You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PeriodicExecutor starts non-daemon threads in subinterpreters, where daemon threads are disallowed. The monitor shutdown handler is registered with threading._register_atexit (only in those interpreters) so executors are stopped and joined at interpreter teardown; main-interpreter shutdown ordering is unchanged.
Added tests for PeriodicExecutor in a subinterpreter and a dedicated test_subinterpreters.py covering direct subinterpeter and InterpreterPoolExecutor usage with sync and aync clients.
Note: an interpreter may block on teardown until in-flight monitor work completes.
Almost all of the changes are new tests.
Test Plan
The new subinterpreter tests pass on Python 3.14 and 3.15.
With the shutdown changes reverted, test_subinterpreters fails, demonstrating regression coverage.
Checklist
Checklist for Author
Did you update the changelog (if necessary)?
Is there test coverage?
Is any followup work tracked in a JIRA ticket? If so, add link(s). (PYTHON-6114 Adds support for testing mod_wsgi 6.0, which requires full support for subinterpreters).
Checklist for Reviewer
Does the title of the PR reference a JIRA Ticket?
Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
Is all relevant documentation (README or docstring) updated?
PeriodicExecutor now falls back to a non-daemon thread when daemon threads are disallowed (subinterpreters). The monitor shutdown handler is also registered with threading._register_atexit so executors are stopped and joined before interpreter teardown.
Adds regression tests for PeriodicExecutor in a real subinterpreter and for concurrent MongoClients across subinterpreters (Python 3.14+).
Runs live MongoClients inside interpreters managed by the standard
InterpreterPoolExecutor, mirroring the existing multi-threaded checks.
The pool's interpreters do not allow daemon threads, so this exercises
the non-daemon monitor thread fallback and its shutdown handling.
The worker interpreters may not have the repo root on sys.path (the
stdlib test package shadows the repo's), so the pickled worker function
failed to unpickle. Submit the builtin exec with a code string that
inserts the main interpreter's sys.path before importing pymongo.
Runs live AsyncMongoClients inside interpreters managed by
InterpreterPoolExecutor. The async client runs its background tasks on
the interpreter's own event loop rather than in threads, covering the
other shutdown path.
Invert the is_running guard in test_subinterpreters: closing an idle
interpreter runs threading._shutdown, which stops and joins pymongo's
monitor threads, so the successful path must close them. Note in the
changelog that both clients are covered by the subinterpreter tests.
Only register the monitor shutdown handler with
threading._register_atexit when the interpreter disallows daemon
threads, preserving normal main interpreter shutdown ordering.
Test AsyncPeriodicExecutor in a subinterpreter in the async suite,
tighten the code comments, and note that both clients are supported in
the changelog.
Add test/asynchronous/test_subinterpreters.py with the subinterpreter
and InterpreterPoolExecutor client tests, written once in async style
and mirrored by synchro. Remove them from test_threads.py, which is
not mirrored. Collapse the periodic executor test's duplicated worker
blocks into one block and make its async target a coroutine.
…preter
The sync worker blocks on an event set by the target's first run, so
the interpreter is destroyed with the monitor live in its interval
loop. The async worker yields to its loop once, which runs the monitor
task's first step.
A failed assertion left the non-daemon worker threads running, which
blocked the test process on exit and leaked the running interpreters.
Join the workers in the cleanup path before closing the interpreters.
The queue read in the subinterpreter worker and the pool future waits
blocked the event loop, freezing the client tasks under test. The
async suite waits through asyncio.to_thread instead, while the sync
suite keeps the direct calls.
…y blocks
Test that PeriodicExecutor.open() starts a non-daemon thread when the
daemon assignment raises, and exclude the monitor shutdown registration
blocks, which only run in the sync twin (the async source's block is
dead code).
The result_stmt interpolation landed outside the worker string as a
set literal, so the pool's futures were never awaited and worker
exceptions were silently ignored. Call future.result() explicitly,
through asyncio.to_thread in the async suite.
_get_n() performs queue.get(timeout=30) synchronously, so this async test blocks its event loop while waiting for subinterpreters. Offload the call with await asyncio.to_thread(...) in the async path and preserve the direct call under _IS_SYNC; apply the same fix to the done wait below.
Offload thread joins from the async event loop
test/asynchronous/test_subinterpreters.py:134
thread.join(60) blocks the async test's event-loop thread. If a subinterpreter is slow or stuck, this prevents the loop from making progress for the whole timeout; await the join through asyncio.to_thread (and apply the same treatment to the cleanup join below).
This issue also appears on line 147 of the same file.
Avoid blocking event loop during executor shutdown
test/asynchronous/test_subinterpreters.py:196
Although each future.result() is offloaded below, exiting this synchronous context manager calls shutdown(wait=True) on the event-loop thread. Worker-interpreter teardown can wait for in-flight monitor work, so this can still block the async test loop for a long time. Create the executor explicitly and, in the async path, await asyncio.to_thread(executor.shutdown) from finally (while retaining direct shutdown in the generated sync path).
The result_stmt interpolation landed outside the worker string as a
set literal, so the pool's futures were never awaited and worker
exceptions were silently ignored. Call future.result() explicitly,
through asyncio.to_thread in the async suite.
Offload the queue waits, thread joins, and pool shutdown to worker
threads in the async suite, where they would otherwise block the event
loop for up to their full timeouts. The sync suite keeps the direct
calls.
The Windows proactor loop cannot be created in a subinterpreter:
signal.set_wakeup_fd only works in the main interpreter. Run the async
workers on a selector loop there instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PYTHON-5418
Changes in this PR
PeriodicExecutorstarts non-daemon threads in subinterpreters, where daemon threads are disallowed. The monitor shutdown handler is registered withthreading._register_atexit(only in those interpreters) so executors are stopped and joined at interpreter teardown; main-interpreter shutdown ordering is unchanged.PeriodicExecutorin a subinterpreter and a dedicatedtest_subinterpreters.pycovering direct subinterpeter andInterpreterPoolExecutorusage with sync and aync clients.Test Plan
test_subinterpretersfails, demonstrating regression coverage.Checklist
Checklist for Author
Checklist for Reviewer