Skip to content

Instantly share code, notes, and snippets.

@ynkdir
Created December 29, 2024 09:23
Show Gist options
  • Save ynkdir/1e7006435a4d05e4c24412cfc5cd7a09 to your computer and use it in GitHub Desktop.
Save ynkdir/1e7006435a4d05e4c24412cfc5cd7a09 to your computer and use it in GitHub Desktop.
Result of asyncio.create_task() may get garbage collected

asyncio.create_task() documentation says

asyncio.create_task(coro, *, name=None, context=None)
...
Important: Save a reference to the result of this function, to avoid a task
disappearing mid-execution. The event loop only keeps weak references to
tasks. A task that isn’t referenced elsewhere may get garbage collected at
any time, even before it’s done. For reliable “fire-and-forget” background
tasks, gather them in a collection:
background_tasks = set()

for i in range(10):
    task = asyncio.create_task(some_coro(param=i))

    # Add task to the set. This creates a strong reference.
    background_tasks.add(task)

    # To prevent keeping references to finished tasks forever,
    # make each task remove its own reference from the set after
    # completion:
    task.add_done_callback(background_tasks.discard)

Reported issue is

shield() documentation should mention user needs to keep reference to the task #94972

Reproducible code is

import weakref
import asyncio
import gc

import logging


logger = logging.getLogger(__name__)


async def async_fn():
    try:
        await asyncio.get_running_loop().create_future()  # (*1)
    except BaseException:
        logger.exception("closed!")
        raise


async def amain():
    weak_task = weakref.ref(asyncio.create_task(async_fn()))
    await asyncio.sleep(0.01)
    gc.collect()  # (*2)
    print(weak_task())


asyncio.run(amain())

Result:

Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<async_fn() running at C:\tmp\a.py:13> wait_for=<Future pending cb=[Task.task_wakeup()]>>
closed!
Traceback (most recent call last):
  File "C:\tmp\a.py", line 13, in async_fn
    await asyncio.get_running_loop().create_future()
GeneratorExit
None

At line (*1), asyncio setup awaiting context as following.

future.add_done_callback(task.__wakeup)
task._fut_waiter = future

Awaiting object is hold by task. And task is hold by _current_tasks

asyncio.tasks._current_tasks -> async_fn -> future -> async_fn

At line (*2), current task was switched to amain(). And no object refer async_fn task. Then, async_fn task and future are garbage collected.

I think that this should not be a problem in normal use case. Because we have to hold future object to call set_result() in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment