Async HTTP Clients: aiohttp vs httpx

Feb 22, 2024 ยท 4 min read

As Python developers, we often need to make HTTP requests to access APIs and web services. When building asynchronous applications, the choice of HTTP client library is important. Two popular async HTTP client libraries for Python are aiohttp and httpx. In this article, we'll compare these two libraries and look at when you may prefer one over the other.

Overview

aiohttp is a well-established HTTP client library that has been around since 2014. It supports both HTTP/1.1 and HTTP/2, and can be used for both client and server code.

httpx is a newer HTTP client that builds on the learnings from requests and aiohttp. It focuses exclusively on the client side, providing an easy to use client API. httpx supports both HTTP/1.1 and HTTP/2, and has a number of handy features for building robust HTTP clients.

Key Differences

Here are some of the key differences between aiohttp and httpx that may influence which one you choose:

  • Client vs client/server - aiohttp supports both client and server use cases, whereas httpx focuses exclusively on the client side.
  • HTTP/2 support - Both libraries have HTTP/2 support, but httpx prioritizes HTTP/2 features more heavily.
  • API design - httpx aims for an easier to use and more Pythonic API than aiohttp.
  • Timeouts - httpx has simpler timeout logic using Python's standard Timeout context manager.
  • Proxy support - httpx has built-in proxy support including authentication.
  • Limits - httpx does not impose limits on the number of connections, whereas aiohttp limits concurrent connections.
  • Dependencies - httpx brings in fewer dependencies than aiohttp.
  • So in summary, httpx focuses on being an easy-to-use, robust and full-featured HTTP client, whereas aiohttp is a more low-level client/server library optimized for performance.

    Example Usage

    Here is some example code showing a simple GET request with both libraries:

    import aiohttp
    import httpx
    
    async def aiohttp_get(url):
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as response:
                return response 
    
    async def httpx_get(url):
        async with httpx.AsyncClient() as client:
            response = await client.get(url)
            return response  

    As you can see, httpx aims for a simpler and more Pythonic API.

    Features

    Let's explore some of the features of these libraries in more depth:

    Timeouts

    httpx uses Python's standard Timeout context manager for timeout logic:

    with httpx.Timeout(1.0):
       response = client.get(url)

    Whereas aiohttp has custom timeout implementations. This makes httpx's timeout handling easier to reason about.

    HTTP/2 Support

    Both libraries support HTTP/2, but there are some differences:

  • httpx prioritizes HTTP/2 features and performance more heavily.
  • aiohttp takes a more conservative approach, falling back to HTTP/1.1 when issues emerge.
  • So if you need full HTTP/2 support, httpx is likely the better choice.

    Proxy Support

    httpx provides built-in proxy support, including authentication. This makes it straightforward to make requests through a proxy:

    client = httpx.AsyncClient(proxies="http://user:pass@10.10.1.10:3128/")

    aiohttp requires additional packages like aiohttp-proxy to provide this functionality.

    Limits

    aiohttp imposes limits on things like the maximum number of concurrent connections, to avoid resource exhaustion.

    In contrast, httpx does not impose any limits, so you need to be careful to avoid overloading resources. But this does make httpx simpler for the common case when you are making a low/moderate number of requests.

    Performance

    Both aiohttp and httpx have excellent performance for an async HTTP client. Some benchmarks have shown httpx to have faster HTTP/1.1 performance than aiohttp in some scenarios.

    However performance should not be the only consideration - make sure to evaluate each library against your specific requirements.

    Conclusion

    The choice between aiohttp and httpx ultimately depends on your specific needs.

    Some key points:

  • If you need an HTTP client/server library, choose aiohttp.
  • If you want the easiest to use client API, choose httpx.
  • If you need extensive HTTP/2 support, lean towards httpx.
  • If you are building a complex application and need control over limits and configuration, aiohttp may be preferable.
  • The good news is that both are excellent choices, so you can't go wrong. Evaluate each against your specific application's needs.

    I hope this gives you a good overview of how these two popular async HTTP clients compare!

    Browse by tags:

    Browse by language:

    Tired of getting blocked while scraping the web?

    ProxiesAPI handles headless browsers and rotates proxies for you.
    Get access to 1,000 free API credits, no credit card required!