Stories from the Web Crawling trenches in asynchronous web applications

Handling Timeouts Gracefully with aiohttp in Python

Author: Mohan Ganesan

Date: Feb 22, 2024

When building asynchronous web applications and APIs in Python with aiohttp, properly handling timeouts is essential. Use ClientTimeout to configure request timeouts and wrap requests in try/except blocks to catch ClientTimeout. Configure global timeout on aiohttp servers with timeout parameter.

Fixing aiohttp UnicodeDecodeErrors

Author: Mohan Ganesan

Date: Mar 3, 2024

Fixing UnicodeDecodeErrors in aiohttp: specify encoding, check actual encoding, decode manually, re-encode text

Handling Errors Gracefully in aiohttp with errors=ignore

Author: Mohan Ganesan

Date: Mar 3, 2024

errors='ignore' prevents aiohttp client errors from crashing your application. Customize exactly which errors to ignore and handle them programmatically. Vital for robust and resilient asynchronous services.

Efficiently Handling Data with aiohttp in Python

Author: Mohan Ganesan

Date: Feb 22, 2024

The aiohttp library is a powerful tool for building asynchronous web applications and APIs in Python. It provides useful abstractions and tools for handling data effectively, including fetching data asynchronously, working with request data, and managing application state.

Building Asynchronous Web APIs with aiohttp Views

Author: Mohan Ganesan

Date: Mar 3, 2024

The aiohttp library in Python provides tools for building asynchronous web applications. A key component is aiohttp views, which allow you to write handler functions for incoming requests similarly to how you would with a traditional web framework like Flask or Django.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!