Stories from the Web Crawling trenches in network programming

How to Build a Simple HTTP Proxy in Rust in just 40 lines

Author: Mohan Ganesan

Date: Oct 1, 2023

Rust is a great language for network programming. Learn how to build a basic HTTP proxy in just 40 lines of code. Also, discover the benefits of using a rotating proxy to avoid IP blocking.

What is the difference between socket and Urllib?

Author: Mohan Ganesan

Date: Feb 8, 2024

Sockets offer low-level network access, but can be complex. urllib makes HTTP requests simple, but with less flexibility.

Achieving Speed with Asyncio in Python

Author: Mohan Ganesan

Date: Mar 24, 2024

Python's asyncio library enables concurrency for improved performance, but not parallelism. It allows efficient use of I/O resources within a single thread.

How to Build a Super Simple HTTP Proxy in R in just 20 lines of code

Author: Mohan Ganesan

Date: Oct 1, 2023

Build a basic HTTP proxy server in R using httpuv and httr packages. Learn how to handle IP blocking with a rotating proxy service.

Leveraging Sockets for Effective Network Communication in Python

Author: Mohan Ganesan

Date: Feb 20, 2024

Sockets in Python enable low-level network communication, providing bidirectional communication, support for multiple protocols, portability, and an accessible API.

What is a socket in Python?

Author: Mohan Ganesan

Date: Feb 20, 2024

Sockets are a key concept in network programming that allow communication between processes or applications. In Python, sockets are enabled through the socket library. Client sockets are used to initiate communication with a server, while server sockets listen for incoming connections. Sockets enable bidirectional communication through sending and receiving data, and can handle multiple client connections concurrently.

Making Python Asynchronous: An Introduction to asyncio

Author: Mohan Ganesan

Date: Mar 17, 2024

Asynchronous programming in Python with asyncio allows for concurrent execution, improved speed and efficiency. It is useful for network programming and database access.

Asyncio events

Author: Mohan Ganesan

Date: Mar 25, 2024

Asyncio is a powerful feature in Python that allows you to write asynchronous, non-blocking code. It enables more responsive programs for I/O bound tasks like web scraping and network programming.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!