Stories from the Web Crawling trenches in concurrent connections

Efficiently Sending Files with aiohttp in Python

Author: Mohan Ganesan

Date: Mar 3, 2024

Sending files over the network asynchronously in Python using aiohttp library for efficient file transfers.

Serving HTTP Requests Efficiently with aiohttp's TCPServer

Author: Mohan Ganesan

Date: Mar 3, 2024

The aiohttp Python library provides powerful tools for building asynchronous HTTP services. TCPServer is a key component that handles details like accepting connections, reading/writing data, and closing connections. It supports HTTPS, handles concurrent connections efficiently, and is useful for microservices and API backends.

Rate Limiting Requests with aiohttp

Author: Mohan Ganesan

Date: Feb 22, 2024

Prevent abuse and reduce server load by rate limiting requests using aiohttp's ThrottleConcurrency middleware.

Build High Performance Asyncio Web Servers in Python

Author: Mohan Ganesan

Date: Mar 25, 2024

Python's asyncio module allows you to write non-blocking, event-driven network code. This makes it possible to build very high performance web servers that can handle thousands of concurrent connections with very low resource usage.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!