Using blocking portal with a loitering event loop thread

example.py

import asyncio
import atexit
from contextlib import ExitStack

from anyio.from_thread import start_blocking_portal

exit_stack = ExitStack()
portal = exit_stack.enter_context(start_blocking_portal())
atexit.register(exit_stack.close)

portal.call(asyncio.sleep, 1)

If I try using this (with either asyncio or trio) my tests work, but pytest hangs at the end. Pressing Ctrl-C gives the below traceback:

<snip>
======================== 5 passed in 1.15s ========================
^CException ignored in: <module 'threading' from '/opt/python/envs/dev310/lib/python3.10/threading.py'>
Traceback (most recent call last):
  File "/opt/python/envs/dev310/lib/python3.10/threading.py", line 1537, in _shutdown
    atexit_call()
  File "/opt/python/envs/dev310/lib/python3.10/concurrent/futures/thread.py", line 31, in _python_exit
    t.join()
  File "/opt/python/envs/dev310/lib/python3.10/threading.py", line 1096, in join
    self._wait_for_tstate_lock()
  File "/opt/python/envs/dev310/lib/python3.10/threading.py", line 1116, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt: 

Simply running the example.py also hangs for me:

❯ python example.py 
^CException ignored in: <module 'threading' from '/opt/python/envs/dev310/lib/python3.10/threading.py'>
Traceback (most recent call last):
  File "/opt/python/envs/dev310/lib/python3.10/threading.py", line 1537, in _shutdown
    atexit_call()
  File "/opt/python/envs/dev310/lib/python3.10/concurrent/futures/thread.py", line 31, in _python_exit
    t.join()
  File "/opt/python/envs/dev310/lib/python3.10/threading.py", line 1096, in join
    self._wait_for_tstate_lock()
  File "/opt/python/envs/dev310/lib/python3.10/threading.py", line 1116, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt: 

I only tested this on Python 3.8, and it seems to hang on any later version.

A bug in CPython?

I think I see the problem. start_blocking_portal() is using a concurrent.futures.ThreadPoolExecutor behind the scenes, and these started using non-daemonic threads in Python 3.9, so atexit callbacks aren't run, leading to a deadlock. I'll need to make it use plain daemonic Thread instead for consistency.

I think I see the problem. start_blocking_portal() is using a concurrent.futures.ThreadPoolExecutor behind the scenes, and these started using non-daemonic threads in Python 3.9, so atexit callbacks aren't run, leading to a deadlock. I'll need to make it use plain daemonic Thread instead for consistency.

Nothing is ever as easy as it should be! 😅 I'm glad you could see the root cause as debugging that was definitely outside my experience/expertise!