What is the difference between asyncio and queue?

Mar 24, 2024 ยท 4 min read

Asynchronous programming has become increasingly popular in Python. The asyncio module provides infrastructure for writing asynchronous code using the async/await syntax. Meanwhile, queues remain a useful tool for coordinating work between threads or processes. This article explains the key differences between asyncio and queues and when to use each in Python.

Async IO Overview

The asyncio module allows you to write code in an asynchronous, non-blocking style using async/await. Here is a quick overview:

  • An async function runs synchronously until it hits an await expression, which pauses it and allows other tasks to execute until that awaitable completes.
  • The asyncio event loop runs these tasks and callbacks concurrently, suspending and resuming them at the await expressions.
  • This makes it possible to have several tasks share the same thread cooperatively rather than blocking it.
  • For I/O bound applications like web servers, asyncio allows handling many concurrent connections efficiently without complex multi-threaded code.

    Here is some example asyncio code:

    import asyncio
    async def fetch_data():
        print('start fetching')
        await asyncio.sleep(2) # simulate waiting for I/O 
        print('done fetching')
        return {'data': 1}
    async def print_numbers():
        for i in range(10):
            await asyncio.sleep(0.25)
    async def main():
        task1 = asyncio.create_task(fetch_data())
        task2 = asyncio.create_task(print_numbers())
        value = await task1
        await task2

    This concurrently runs fetch_data and print_numbers allowing other tasks to execute while one awaits.

    Queue Overview

    A queue data structure allows threads or processes to safely exchange data. One or more producers add data to the queue, and one or more consumers take data out of the queue. The queue handles the synchronization and buffering between them.

    The queue module provides synchronized queue classes. Here is an example of using a queue from concurrent threads:

    from threading import Thread 
    from queue import Queue
    def worker(queue):
        while True:
            item = queue.get()
            print(f'Working on {item}')
            print(f'Finished {item}')
    q = Queue()
    for i in range(10):
         t = Thread(target=worker, args=(q,))
         t.daemon = True
    for item in range(20):

    This allows coordinating work between several threads, with the Queue handling the synchronization.

    Key Differences

    Now that we have looked at their basic usage, what are the main differences between asyncio and queues?

    Concurrency Model

    The key difference is their concurrency model:

  • Asyncio uses cooperative multitasking in a single thread. At the await expressions, tasks yield control back to the event loop, allowing other tasks to run while waiting for I/O.
  • Queues coordinate work across multiple threads or processes. Producers and consumers run independently, with the queue mediating transfer between them.
  • So asyncio enables concurrency through coroutines in a single thread, while queues use separate threads/processes.

    Use Cases

    Consequently, here are the typical use cases for each:

  • Asyncio is ideal for I/O bound tasks like web servers, allowing handling many connections concurrently. The single thread can juggle many tasks.
  • Queues shine for CPU bound work like processing data, simulations or machine learning. Different threads/processes allow utilizing multiple CPU cores for parallel processing.
  • For I/O bound applications that need scalability, asyncio is usually the best approach in Python. But queues still serve an important role in coordinating CPU bound processes across threads or servers.


    In terms of performance:

  • Asyncio has very low overhead for switching between tasks or awaiting I/O. But it runs on a single core without parallelism.
  • Queues allow utilizing multiple processes and cores, but have higher coordination overhead than asyncio.
  • So there is a CPU/throughput tradeoff - asyncio provides very lightweight concurrency, while queues enable more parallelism.


    Finally, the granularity levels differ:

  • Asyncio coordinates at a coroutine/task level for finer grained control flow.
  • Queues coordinate at a process/thread level which is more heavyweight.
  • So asyncio allows managing concurrency at a more fine-grained level.

    When to Use Each

    Based on their differences, here is guidance on when to use asyncio vs queues:

  • For I/O bound apps like web/RPC servers, asyncio is ideal - it enables handling many concurrent connections in a lightweight way.
  • For CPU intensive work like data processing or computations, queues enable utilizing multiple processes across cores for better parallelism.
  • For finer grained coordination of concurrent tasks, asyncio provides lightweight control flow.
  • For heavier coordination across processes or threads, queues do the job.
  • So in summary:

  • Asyncio for I/O concurrency, queues for parallelism.
  • Asyncio for control flow, queues for processes.
  • Hopefully this gives you a better sense of how to decide between using asyncio or queues in Python! Each has strengths in particular use cases and concurrency models.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: