Handling Errors Gracefully in aiohttp with errors=ignore

Mar 3, 2024 ยท 3 min read

When building asynchronous web applications and APIs with the popular Python aiohttp library, properly handling errors is crucial for robustness and reliability. By default, aiohttp will raise exceptions on errors which will crash your application if unhandled. However, for certain non-critical errors, we may want to handle them more gracefully to keep the application running rather than crashing. That's where the errors=ignore parameter can help.

What does errors=ignore do?

Passing errors='ignore' when creating an aiohttp ClientSession will prevent exceptions from being raised on certain client-side errors. Instead, the errors will be returned in the response object allowing you to handle them programmatically. For example:

import aiohttp

async with aiohttp.ClientSession(errors='ignore') as session:
    async with session.get('http://invalid-host') as resp:
        print(resp.status) # Prints 502

Rather than crashing on the DNS lookup failure for an invalid host, a response is still returned with status 502. This allows catching the error and handling it appropriately in your application code.

When should you use it?

errors=ignore is useful when making calls to external services that can fail intermittently and are not critical to the core application flow. For example, failing to call an analytics API should not bring down the whole application. Other good use cases include:

  • Calling non-essential microservices and APIs
  • Enqueuing background jobs and tasks
  • Logging metrics and analytics events
  • However, it's important not to overuse errors=ignore as ignoring too many failures can hide real problems in your application. Use it judiciously only for non-essential operations.

    What errors get ignored?

    By default, the following errors will be ignored and returned as a client response:

  • DNS lookup failures
  • Connection failures and timeouts
  • Too many redirects
  • Certain SSL errors
  • However other exceptions like those caused by invalid HTTP syntax or application code errors will still raise exceptions.

    See the full list of ignored exceptions in the aiohttp documentation.

    Customizing ignored errors

    You can customize what errors get ignored by passing an Exception class or tuple of classes to ignore:

    from aiohttp import ClientConnectorCertificateError
    async with aiohttp.ClientSession(errors=(ClientConnectorCertificateError,)) as session:
        # Ignores only SSL certificate errors

    This gives more control over which specific errors to handle gracefully while letting others fail loudly.

    Handling ignored errors

    When an error is ignored, the response object will contain details about the failure:

    resp = await session.get('http://invalid-host')
    print(resp.status) # 502 
    print(resp.content_type) # None
    print(resp.content) # empty byte string
    print(resp.exception()) # Returns the ConnectorError exception object

    You can check the response attributes like status code, headers, and content to identify the failure, log it, and handle it appropriately.

    The key is to anticipate and design your application flows to gracefully capture and recover from expected errors by inspecting the response. Robust error handling is essential for building reliable asynchronous services.

    In Summary

  • errors='ignore' prevents aiohttp client errors from crashing your application
  • Perfect for handling intermittent failures in non-critical paths
  • Customize exactly which errors to ignore
  • Inspect the response object to handle the errors programmatically
  • Vital for robust and resilient asynchronous services
  • Handling errors gracefully is essential for any application interacting with unreliable networks. aiohttp's errors='ignore' parameter gives developers a useful tool for building resilient non-blocking services that can anticipate and recover from intermittent failures.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: