Stories from the Web Crawling trenches in CPU-bound tasks

Speed Up Your Asyncio Code with Thread Pools

Author: Mohan Ganesan

Date: Mar 25, 2024

Asyncio is great for writing non-blocking network code in Python. But sometimes you have CPU-bound tasks that could benefit from parallel execution. That's where thread pools come in handy!

Python Threads vs Processes: Which is Faster and When to Use Each

Author: Mohan Ganesan

Date: Mar 24, 2024

When writing Python programs, developers often wonder if it's better to use threads or processes. Processes are generally faster and more robust, but have higher overhead. Threads require less resources to create, but come with their own challenges.

Does Python asyncio use threads?

Author: Mohan Ganesan

Date: Mar 24, 2024

Python's asyncio module provides single-threaded concurrency using coroutines and an event loop. It can offload blocking IO and CPU-bound tasks to thread pools.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!