gh-144356: Avoid races when computing set_iterator.__length_hint__ under no-gil
#144357
Conversation
setiter_len() was reading so->used without atomic access while concurrent mutations update it atomically under Py_GIL_DISABLED. Use an atomic load for so->used to avoid a data race. This preserves the existing semantics of __length_hint__ while making the access thread-safe. Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
| for t in threads: | ||
| t.start() | ||
|
|
||
| stop.set() |
There was a problem hiding this comment.
This means the threads will stop right after they have started. I would prefer the pattern that is used in some other tests in this file: set a constant NUM_LOOPS (determined so that the test < 0.1 seconds, but there still is a decent number of mutations)
Objects/setobject.c
Outdated
| setiterobject *si = (setiterobject*)op; | ||
| Py_ssize_t len = 0; | ||
| if (si->si_set != NULL && si->si_used == si->si_set->used) | ||
| PySetObject *so = si->si_set; |
There was a problem hiding this comment.
Here so is a borrowed reference to si->si_set. But si->si_set can be cleared in setiter_iternext (if the iterator is exhausted) outside the critical section.
This is a different mechanism than the corresponding issue, so maybe something to address in another PR. But solving both together is something to consider.
There was a problem hiding this comment.
Good catch. Thanks a lot.
hyongtao-code
changed the title
setiter_len() under no-gilset_iterator.__length_hint__ under no-gil
|
Thanks for the review. I’ve decided to address both issues in this PR. I also added a corresponding test case for the issue you pointed out. |
2 participants
Long log:
setiter_len()was readingso->usedwithout atomic access while concurrentmutations update it atomically under Py_GIL_DISABLED.
In free-threaded builds,
setiter_len()could race with concurrent setmutation and iterator exhaustion.
Use an atomic load for
so->usedto avoid a data race. This preserves theexisting semantics of
__length_hint__while making the access thread-safe.Signed-off-by: Yongtao Huang yongtaoh2022@gmail.com