Stories from the Web Crawling trenches in processes

Making Asynchronous HTTP Requests in Python without Waiting for a Response

Author: Mohan Ganesan

Date: Feb 3, 2024

Make asynchronous HTTP requests in Python without blocking using the requests library, asyncio module, or threads/processes.

Making Concurrent Requests in Python: A Programmer's Guide

Author: Mohan Ganesan

Date: Nov 18, 2023

Handling multiple API calls and web scraping concurrently is critical for Python developers. This guide explores techniques for performant concurrent requests in Python.

Python Threads vs Processes: Which is Faster and When to Use Each

Author: Mohan Ganesan

Date: Mar 24, 2024

When writing Python programs, developers often wonder if it's better to use threads or processes. Processes are generally faster and more robust, but have higher overhead. Threads require less resources to create, but come with their own challenges.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!