Skip to content

gh-123471: Make concurrent iteration over itertools.pairwise thread-safe#144489

Open
eendebakpt wants to merge 6 commits intopython:mainfrom
eendebakpt:pairwise_ft_v3
Open

gh-123471: Make concurrent iteration over itertools.pairwise thread-safe#144489
eendebakpt wants to merge 6 commits intopython:mainfrom
eendebakpt:pairwise_ft_v3

Conversation

@eendebakpt
Copy link
Contributor

@eendebakpt eendebakpt commented Feb 4, 2026

We use a critical section similar to the approach in #132814. The ideas from an earlier PR #125417 are too complex (or not sufficient for thread-safety).

We also avoid an incref/decref pair.

@rhettinger
Copy link
Contributor

rhettinger commented Feb 5, 2026

I'm thinking that this isn't the right strategy for itertools. Instead, we should implement the same semantics as for generators which is to raise an exception if a concurrent call occurs.

If a user wants concurrency for either generators or itertools, then they with need to wrap the iterators with serialize or a concurrent version of tee as discussed in the planning issue.

@eendebakpt
Copy link
Contributor Author

@rhettinger I agree with you that concurrent iteration over an iterator from itertools is probably not what the user wants and having to wrap in serialize is a minor inconvenience to make concurrent iteration possible.

However, the change you propose is not backwards compatible. Do you want to start a discussion on DPO for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants