The mysterious concurrent.futures._base.CancelledError out of nowhere

When you’re messing with Python 3.7’s asyncio.Task and cancellation, you might come across a weird Exception in your main loop like this:

Traceback (most recent call last):
  File "/scratch.py", line 16, in 
    asyncio.run(main())
  File "/usr/lib64/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib64/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
concurrent.futures._base.CancelledError

The example code I used to generate that exception is:


import asyncio

async def a():
    asyncio.current_task().cancel()
    await asyncio.sleep(0)
    # Upon return of control, CancelledError is injected here

async def main():
    t = [a(), a()]
    await asyncio.gather(*t)

asyncio.run(main())

As you can see, we have a list of asyncio.Task that we asyncio.gather(). The intention of this code is for a coroutine to cancel itself such that the awaiter (whoever called asyncio.gather()) can find out whether the asyncio.Task completed successfully or was cancelled.

The reason this snippet of code does not work is because by default, neither asyncio.gather() nor asyncio.Task will catch CancelledError for you. This means that once all the asyncio.gather() awaitables have completed, then you’ll get the first error that occurred, in this case the CancelledError.

The moral of the story is, don’t raise CancelledError within an awaitable. asyncio.Task.cancel() is supposed to be used to inject a termination into an asyncio function, not to be used within it.

If you have to do it, then you better either use return_exceptions=True or use a wrapper to try-catch the CancelledError.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s