Stories from the Web Crawling trenches in data flow

Async IO and Generators: Key Differences in Python

Author: Mohan Ganesan

Date: Mar 24, 2024

Async IO and generators are powerful asynchronous programming concepts in Python with key differences. Generators produce data on demand, while Async IO enables concurrent work. Both are useful for different scenarios and can be used together to write highly scalable programs.

Efficiently Handling Data with aiohttp in Python

Author: Mohan Ganesan

Date: Feb 22, 2024

The aiohttp library is a powerful tool for building asynchronous web applications and APIs in Python. It provides useful abstractions and tools for handling data effectively, including fetching data asynchronously, working with request data, and managing application state.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!