Stories from the Web Crawling trenches in status code

Handling HTTP Status Codes with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

When making HTTP requests in Python, it's important to check the status code of the response. The requests library makes this easy, allowing you to handle success and error codes correctly.

Handling 404 Errors when Making HTTP Requests in Python

Author: Mohan Ganesan

Date: Feb 3, 2024

Tips on handling 404 errors gracefully in Python code when making HTTP requests. Check response status code, log and notify, use try-except block.

Handling Errors with aiohttp ClientResponseError

Author: Mohan Ganesan

Date: Feb 22, 2024

Handle aiohttp ClientResponseError in Python for robust and user-friendly applications.

Troubleshooting Python Requests Get When Webpage Isn't Loading

Author: Mohan Ganesan

Date: Feb 3, 2024

When using Python's Requests library to load a webpage, troubleshoot by checking the URL, status code, response headers.

Handling HTTP Response Codes with Python's urllib

Author: Mohan Ganesan

Date: Feb 8, 2024

Check HTTP response codes in Python using urllib. Get the response code and reason phrase to understand the outcome of web requests.

Handling Client Errors with aiohttp

Author: Mohan Ganesan

Date: Feb 22, 2024

When building applications with aiohttp, it is important to handle client errors properly. Use the ClientResponseError exception and status code to identify client errors and implement custom error handling logic for expected cases.

Fetching Data from APIs with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Accessing data from web APIs using Python's Requests library. Learn how to make GET requests, process responses, and handle errors.

Handling Responses with urllib in Python

Author: Mohan Ganesan

Date: Feb 6, 2024

The urllib module in Python provides functionality for fetching data from URLs. Properly handling the response is important for robust code.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!