Stories from the Web Crawling trenches in httpx

Getting Started with HTTPX in Python: Practical Examples and Usage Tips

Author: Mohan Ganesan

Date: Feb 5, 2024

HTTPX is a powerful Python HTTP client that makes API calls, handles authentication, timeouts, and more. Easily make GET and POST requests, handle JSON, forms, files, and headers. Supports async requests and session reuse for optimal performance.

Persisting Sessions with Httpx in Python

Author: Mohan Ganesan

Date: Feb 5, 2024

Guide on utilizing Httpx's session support to maintain state and persist cookies across multiple requests in Python.

Sending POST Data with HTTPX in Python

Author: Mohan Ganesan

Date: Feb 5, 2024

HTTPX is a popular Python library for making HTTP requests. This guide explains how to properly structure and send POST data with HTTPX.

Speed Up Web App Testing with HTTPX on Kali

Author: Mohan Ganesan

Date: Feb 5, 2024

Kali Linux is a popular penetration testing distribution. HTTPX is a new tool for web application testing. Install it on your Kali box for faster and more efficient web app assessments.

Making HTTP Requests in Python with HTTPX

Author: Mohan Ganesan

Date: Feb 5, 2024

Python HTTP client HTTPX simplifies making HTTP requests, supports HTTP/1.1 and HTTP/2, and offers features like timeouts and retries.

Async HTTP Clients: aiohttp vs httpx

Author: Mohan Ganesan

Date: Feb 22, 2024

Python developers often make HTTP requests to access APIs and web services. Two popular async HTTP client libraries for Python are aiohttp and httpx. This article compares the two libraries and discusses their key differences, features, and performance. The choice between aiohttp and httpx depends on specific needs, such as client/server use cases, HTTP/2 support, ease of use, and control over limits and configuration.

Customizing HTTPX User Agents for Effective API Requests

Author: Mohan Ganesan

Date: Feb 5, 2024

Customize the User Agent header in HTTPX Python library for API analytics, compatibility checks, and access control.

Requests vs urllib vs httpx vs aiohttp

Author: Mohan Ganesan

Date: Feb 3, 2024

Making HTTP requests in Python: comparing Requests, urllib, httpx, and aiohttp. Requests is the easiest, urllib is lower-level, httpx adds advanced features, and aiohttp is for asyncio-based code.

Getting Started with the HTTPX Python Library

Author: Mohan Ganesan

Date: Feb 5, 2024

The HTTPX library is a powerful and user-friendly HTTP client for Python. Install it with pip and make requests easily with its elegant API.

Scraping Websites Without Requests: 4 Python Alternatives

Author: Mohan Ganesan

Date: Feb 3, 2024

The Python Requests module is a popular, easy way to download web pages and scrape data. But what if you need an alternative? Here are 5 good options to scrape websites without Requests.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!