Stories from the Web Crawling trenches in websites

Sending Data in Requests: Payloads, Headers, and Parameters

Author: Mohan Ganesan

Date: Feb 3, 2024

The Python requests module allows you to easily send HTTP requests to APIs and websites. You can attach data as JSON payloads, form-encoded data, or query parameters.

Streamlining HTTP Requests in Python with the Requests Module

Author: Mohan Ganesan

Date: Feb 3, 2024

The Python Requests module is an essential tool for interacting with APIs and websites in your Python code.

What is the difference between web scraping and data scraping?

Author: Mohan Ganesan

Date: Feb 5, 2024

Web scraping extracts data from web pages, while data scraping is a broader term for extracting data from any online source.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!

Try ProxiesAPI for free

Tired of getting blocked while scraping the web?

Get access to 1,000 free API credits, no credit card required!

Try for free