Efficiently Sending Files with aiohttp in Python

Mar 3, 2024 ยท 4 min read

Sending files over the network is a common task in many Python applications. The aiohttp library provides a nice API for handling file transfers asynchronously and efficiently in your Python code.

In this guide, we'll walk through a practical example of sending a file from a Python script to a client using aiohttp. We'll cover some key concepts and best practices along the way.

Why aiohttp for Sending Files?

The aiohttp library is great for building asynchronous Python applications. It shines for tasks like:

  • Creating HTTP servers and clients
  • Handling concurrent connections
  • Transferring data efficiently
  • Compared to alternatives like basic sockets, aiohttp handles a lot of complexity for you. And it allows your Python code to offload blocking I/O work to the background.

    This makes it a great fit for transferring files. The asynchronous design means your code stays responsive while a file sends in the background.

    Sending a File Example

    Let's look at a full example of sending a file with aiohttp.

    First we'll start up an aiohttp server to handle incoming client connections:

    import aiohttp
    import aiofiles
    async def handle_file_request(request):
        file_path = "example_file.pdf"
        async with aiofiles.open(file_path, mode='rb') as f:
            file_data = await f.read()
        return web.Response(
                "Content-Disposition": f'attachment; filename={file_path}'
    app = web.Application()
    app.add_routes([web.get('/', handle_file_request)])
    if __name__ == '__main__':

    The key parts:

  • We open the file to send using aiofiles
  • Read the file data into memory
  • Construct a web.Response with the file bytes as the response body
  • Include a Content-Disposition header to tell the client it's an attachment
  • Now when clients connect and request the root URL path, they'll receive the PDF file.

    The code is clean and simple. But there's still room for improvement...

    Streaming Large Files

    For large files, reading the entire contents into memory can be inefficient.

    We can optimize by streaming chunks of the file instead. This prevents loading gigabytes of file data at once:

    async def stream_file(request):
        file_path = "massive_video.mp4"
        chunk_size = 1024*1024 # 1MB chunks
        async with aiofiles.open(file_path, mode='rb') as f:
            resp = web.StreamResponse(
                    "Content-Disposition": f'attachment; filename={file_path}'
            await resp.prepare(request)
            while True:
                chunk = await f.read(chunk_size)
                if not chunk:
                await resp.write(chunk)
        return resp

    By yielding 1MB chunks instead of the full file, we reduce memory usage. The client will receive the same complete file streamed across multiple response packets.

    And aiohttp handles the asynchronous chunk streaming without blocking our main thread.

    Handling Many Files Efficiently

    What if we need to send multiple files from the same server? Opening each one synchronously would limit performance:

    for file_path in file_paths:
        async with open(file_path) as f:
            await process(f) # sync open

    The trick is to launch all the file reads concurrently:

    import asyncio
    async def stream_files(request):
        file_paths = ["file1.pdf", "file2.mov", "file3.doc"]
        file_chunks = []
        async def read_chunk(fp):
            async with aiofiles.open(fp, 'rb') as f:
                while True:
                    chunk = await f.read(1024*1024) 
                    if not chunk:
        await asyncio.gather(*[read_chunk(fp) for fp in file_paths])
        # Combine file chunks and stream response...

    By gathering the aiofiles read coroutines concurrently, we maximize I/O throughput. The event loop interleaves reads from each file.

    This processes 100+ files much faster than sequential sync opens!

    Other Features

    We've covered the basics of sending files with aiohttp. But there are many other neat features like:

  • Compression - gzip files before sending
  • Range requests - resume large downloads
  • Caching - speed up repeat file requests
  • ETags - enable client-side caching
  • The aiohttp docs show how to enable these.

    Key Takeaways

  • aiohttp provides an async API for sending files from Python
  • For large files, stream chunks instead of loading the full contents
  • Use asyncio.gather to open files concurrently when sending multiples
  • Features like compression, ranges, and caching optimize file transfers
  • Hopefully this gives you a solid starting point for sending files asynchronously in Python.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: