Stories from the Web Crawling trenches in synchronization

Concurrency and Thread Safety in Python's asyncio

Author: Mohan Ganesan

Date: Mar 17, 2024

Python's asyncio module enables concurrency within a single thread using an event loop. Sharing data between coroutines is thread-safe. Multithreading requires new event loops and explicit synchronization. Blocking code must execute in threads to avoid blocking the event loop. Following these best practices ensures efficient, thread-safe asyncio code.

Using Asyncio Conditions for Stateful Coroutines

Author: Mohan Ganesan

Date: Mar 25, 2024

Asyncio conditions allow coroutines to wait for certain states or events during execution. They are useful for scenarios where you need to coordinate or synchronize several coroutines based on shared state.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!