Keeping Data Flowing with aiohttp Streaming Responses

Feb 22, 2024 ยท 3 min read

When building web APIs and other web services with aiohttp, you'll often need to send large chunks of dynamic data to clients. Streaming responses allow you to send this data as it becomes available rather than buffering it all into memory. This brings several advantages like reduced memory usage and getting data to clients faster.

Let's walk through how aiohttp streaming works and some best practices for using it effectively.

Why Streaming Responses?

Before we dig into the details, it's helpful to understand why streaming is useful:

  • Sends data as it's generated - No need to generate an entire large response in memory before sending. This saves memory.
  • Gets data to client faster - Users see data right away rather than waiting for the full response. Better experience.
  • Handles unlimited data - Can stream data of essentially unlimited size from databases, files, etc.
  • Now let's look at how to create streaming responses with aiohttp.

    Creating a Streaming Response

    Using streaming is simple - just return a StreamResponse instead of the usual Response:

    from aiohttp import web
    async def handle(request):
        return web.StreamResponse()

    After creating the response, make sure to call prepare() to initialize headers, status code, etc:

    res = web.StreamResponse()
    res.content_type = "text/plain"
    await res.prepare(request)

    Now you have a prepared streaming response ready to have data written to it!

    Writing Data to the Response

    To stream data, simply write it to the response whenever you have new content available:

    for chunk in generate_data():
        await res.write(chunk.encode())

    A few pointers here:

  • Encode data to bytes using .encode() before writing
  • Data will be sent to client each time write() is called
  • Can be called as many times as needed
  • When done, call write_eof() to finalize the response:

    await res.write_eof()

    And that's the basics of streaming data! Let's look at a full example.

    Putting It All Together

    Here is a full handler that streams random data to show the concepts in action:

    import asyncio
    from aiohttp import web
    import random
    async def handle(request):
        res = web.StreamResponse()
        res.content_type = "text/plain"
        await res.prepare(request)
        for i in range(10):
            data = f"Chunk #{i} {random.randint(1, 100)}\n" 
            await res.write(data.encode())
            await asyncio.sleep(1)
        await res.write_eof()
        return res

    And that's all there is to the fundamentals! Streaming takes care of managing connections, buffers, and data sending automatically.

    There's more we could dig into like compression, chunked transfer encoding, and graceful shutdown handling - but this covers the key basics of getting a stream response set up and sending data.

    Key Takeaways

    To recap streaming responses:

  • Create a StreamResponse
  • Prepare with headers, status, etc
  • Write data chunks using response.write()
  • Call write_eof() when finished
  • Clients receive data as it's written
  • Streaming is great for efficiently piping data through to clients. It reduces resource usage and gets content out faster.

    Give streaming a try next time you need to send large dynamic responses!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl ""

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: