Stories from the Web Crawling trenches in API calls

Working with Query Parameters in Python Requests

Author: Mohan Ganesan

Date: Oct 22, 2023

Query parameters are essential for making API calls and web scraping in Python. Learn how to pass and access query parameters using the Requests library.

Using Proxies with Python Requests

Author: Mohan Ganesan

Date: Oct 22, 2023

Python requests library simplifies HTTP requests and API calls. Proxies help avoid IP blocking. Configure proxies using a dictionary or environment variables. Authenticate requests with credentials. Use sessions for persistent data. Disable SSL verification if trusted. Adjust timeouts and retries for robust requests.

Uploading Images with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Sending images over HTTP requests is a common task in many Python applications. The Requests library provides a simple API for attaching images and other files to POST requests.

Getting Started with HTTPX in Python: Practical Examples and Usage Tips

Author: Mohan Ganesan

Date: Feb 5, 2024

HTTPX is a powerful Python HTTP client that makes API calls, handles authentication, timeouts, and more. Easily make GET and POST requests, handle JSON, forms, files, and headers. Supports async requests and session reuse for optimal performance.

Making Concurrent Requests in Python: A Programmer's Guide

Author: Mohan Ganesan

Date: Nov 18, 2023

Handling multiple API calls and web scraping concurrently is critical for Python developers. This guide explores techniques for performant concurrent requests in Python.

Sending POST Requests with Python's urllib

Author: Mohan Ganesan

Date: Feb 6, 2024

The urllib module in Python provides functionality for sending HTTP POST requests to web servers and handling responses.

Fixing the "Expecting Value" Error with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

When making API calls with the Python Requests library, you may occasionally see the error 'Expecting value', with a 400 status code. This usually means there was an issue with the request data being sent.

Making API Calls with Lists in Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

The Python Requests library provides an easy way to call APIs. You can pass lists of data, like IDs, to be handled by the API. For large lists, join items into a comma separated string to avoid errors.

Boosting Your Discord Bot's Performance with aiohttp

Author: Mohan Ganesan

Date: Feb 22, 2024

Build high-performance Discord bots with aiohttp, the leading asynchronous HTTP client for Python, to prevent blocking and improve concurrency.

Simplifying REST API Calls with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Interacting with REST APIs made easy with Python's Requests module. Simple syntax, JSON decoding, parameterization, and more. Try it now!

Apify Alternative - Why Proxies API is a Simple & Affordable Option

Author: Mohan Ganesan

Date: Sep 30, 2023

Proxies API offers a simpler and more affordable solution to web scraping compared to Apify, with a simple API for HTML scraping and pay-per-call pricing.

How to call API URL?

Author: Mohan Ganesan

Date: May 7, 2024

When building an application, you'll often need to retrieve or send data to an API. This guide covers the basics of making API requests in your code.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!