Stories from the Web Crawling trenches in urllib3

Logging and Debugging with Requests

Author: Mohan Ganesan

Date: Oct 22, 2023

Guide to enable detailed logging and debugging with Requests library in Python for HTTP requests using urllib3 and http.client.

Python's URL Handling Libraries compared - urllib vs requests

Author: Mohan Ganesan

Date: Nov 17, 2023

Python's URL handling libraries have evolved over time, from urllib to urllib2 to urllib3 and finally to requests. Each library offers different features and capabilities, making it important to choose the right one for your needs.

What is PoolManager in urllib3?

Author: Mohan Ganesan

Date: Feb 20, 2024

Simplifying HTTP requests with PoolManager in Python. PoolManager manages a pool of connections for reusing, improving performance. Customize pool behavior for better resource usage.

TLS Support in Python's urllib3

Author: Mohan Ganesan

Date: Feb 8, 2024

urllib3 library supports TLS v1.2 and TLS v1.3 by default, ensuring secure connections in Python. Beware of outdated TLS versions and upgrade urllib3 for security.

Downloading Images from URLs in Python

Author: Mohan Ganesan

Date: May 5, 2024

Learn five different ways to download images from URLs using Python: urllib.request, requests, urllib3, wget, and PyCURL.

Enable Detailed HTTP Debug Logging in Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Enable debug logging in Python Requests library to get detailed insight into HTTP requests and save time debugging issues.

Simplifying HTTP Requests in Python: Requests vs urllib3

Author: Mohan Ganesan

Date: Feb 3, 2024

Making HTTP requests in Python: choose between requests and urllib3. Requests is simple and beginner friendly, while urllib3 offers more control and customization.

Do I need urllib3?

Author: Mohan Ganesan

Date: Feb 20, 2024

Python's urllib3 module provides connection pooling, asynchronous requests, and TLS encryption for better performance in HTTP requests.

Does Python requests use urllib3?

Author: Mohan Ganesan

Date: Feb 20, 2024

Python requests library provides a high-level interface for making HTTP requests, while urllib3 handles the low-level details.

Why use urllib3?

Author: Mohan Ganesan

Date: Feb 20, 2024

urllib3 is a full-featured HTTP client for making requests in Python. It handles connection pooling, SSL/TLS verification, and more, making it a popular choice for web APIs.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!