Stories from the Web Crawling trenches in gather()

Running multiple asyncio tasks

Author: Mohan Ganesan

Date: Mar 25, 2024

When writing async code in Python, asyncio provides two methods for running async tasks in parallel: asyncio.gather() and asyncio.create_task(). gather() bundles tasks and waits for them, while create_task() schedules background work.

How asyncio works in Python?

Author: Mohan Ganesan

Date: Mar 24, 2024

Python's asyncio module allows for writing concurrent code using async/await syntax. It provides an event loop, async functions, and the ability to run awaitables concurrently with asyncio.gather().

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!