Stories from the Web Crawling trenches in response status code

Controlling Redirections in Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Disable auto redirects in Python Requests using allow_redirects=False whenever you want to handle redirects manually.

Automate Website Logins with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Logging into websites made easy with Python's requests module. Replicate login process, handle response codes, automate workflows.

Sending HTTP POST Requests in Python

Author: Mohan Ganesan

Date: Feb 3, 2024

Python provides simple methods to simulate HTTP POST requests for testing APIs or web applications. The main tool for sending HTTP requests in Python is the requests library.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!

Try ProxiesAPI for free

Tired of getting blocked while scraping the web?

Get access to 1,000 free API credits, no credit card required!

Try for free