Asyncio gathering task results

Mar 25, 2024 ยท 3 min read

The asyncio.gather() function is very useful when you need to launch multiple coroutines concurrently and wait for all of their results at once. However, understanding exactly how it works can be tricky for asyncio beginners.

In this guide, we'll walk through real examples to demystify asyncio.gather() and help you use it effectively in your own asyncio code.

The Problem asyncio.gather Solves

Imagine we need to fetch two web pages concurrently using the aiohttp library:

import asyncio
import aiohttp

async def fetch_page(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()''))''))

This launches each coroutine serially, one waiting for the other to finish.

But we want them to run concurrently and process both results when available. That's where gather comes in!

Gather Basics

asyncio.gather() runs coroutines concurrently and blocks until they have all completed:

import asyncio

async def coro1():
    return 1

async def coro2():
    return 2 

async def main():
    results = await asyncio.gather(

# Prints [1, 2]

The key things to know about gather:

  • It accepts multiple coroutines as arguments and awaits them concurrently.
  • It returns results in the order of the coroutines passed in.
  • It raises the first exception encountered, cancelling remaining tasks.
  • This handles our use case of waiting for multiple fetches nicely!

    import asyncio
    import aiohttp
    async def fetch(url):
        async with aiohttp.request('GET', url) as response:
            data = await response.text() 
            return data
    async def main():
        urls = ['', '']
        results = await asyncio.gather(*[fetch(url) for url in urls])
        print(results[0][:100]) # Prints first 100 chars of first page
        print(results[1][:100]) # Prints first 100 chars of second page

    Now both fetches run concurrently, yet we can process their responses sequentially.

    Gathering Tasks That Raise Exceptions

    As mentioned earlier, gather() cancels all pending tasks as soon as one raises an exception.

    Let's see this in action:

    import asyncio
    async def bad_coro():
        raise Exception("Oops!")
    async def main():
            await asyncio.gather(
        except Exception:
            print("Hit exception!") 
    # Prints "Hit exception!"

    Here, our second task gets cancelled even though it wasn't the one that failed.

    So in a real app, we'd typically wrap each task in its own try/except block to isolate failures:

    async def main():
        results = await asyncio.gather(
    async def try_task1():
            return await real_task1() 
        except Exception:
            return None

    This keeps failures isolated and our other tasks can complete.

    Gather Use Cases

    Some examples where gather() shines:

  • Waiting on multiple web requests
  • Waiting on tasks with IO (files, db access, etc)
  • Orchestrating interdependent parallel workflows
  • Throttling task execution to improve performance
  • It's a versatile tool for coordinating independent async work!

    Key Takeaways

  • asyncio.gather() runs coroutines concurrently and aggregates results
  • Results are returned in the order coroutines are passed in
  • If any task raises an exception, all others are cancelled
  • Wrap tasks in try/except blocks to isolate failures
  • Useful for coordinating web requests, IO work, parallel flows
  • I hope these examples help demystify gather() and illuminate how you can apply it. Asyncio has a learning curve, but practice makes perfect!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl ""

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: