Stories from the Web Crawling trenches in async

Benchmarking aiohttp Web Performance

Author: Mohan Ganesan

Date: Feb 22, 2024

The Python aiohttp library provides powerful async HTTP client/server functionality. Benchmarking quantifies metrics like requests per second, latency distributions, and resource usage to guide optimization and capacity planning.

Integrating Peewee ORM with aiohttp for Asynchronous Database Access

Author: Mohan Ganesan

Date: Feb 22, 2024

The aiohttp library provides powerful tools for building asynchronous Python web applications. Peewee is a simple yet powerful ORM for working with SQL databases. Integrating these libraries allows building high-performance async web apps with a Pythonic object-relational mapper for the database access.

Beyond Asyncio: Exploring Asynchronous Programming Options in Python

Author: Mohan Ganesan

Date: Mar 25, 2024

Asyncio is Python's built-in asynchronous programming framework, but there are alternative options like Twisted, Trio, and Curio for non-blocking applications.

When to use async python ?

Author: Mohan Ganesan

Date: Mar 17, 2024

Python developers can use async code for faster and more efficient programming. Async is useful for network requests, file I/O, concurrency, and improving perceived performance. However, it should be avoided for CPU intensive tasks. Mixing async and sync code can cause deadlocks, and debugging async code can be challenging. Bridge between sync and async with asyncio.to_thread() and use purpose-built tools like aiomonitor for debugging.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!