Stories from the Web Crawling trenches in web services

How to install urllib in Python?

Author: Mohan Ganesan

Date: Feb 20, 2024

The urllib module in Python allows easy access to internet data and parsing URLs. It is a must-know module for every Python programmer.

Sending Multipart Form Data with Python's urllib

Author: Mohan Ganesan

Date: Feb 8, 2024

Python's urllib library provides tools to handle multipart form data for integrating with web services. Use requests library to simplify sending multipart form data.

Keeping Sessions Alive with Persistent Connections in Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Using persistent sessions in Python Requests library improves performance and allows reusing connections for multiple requests.

Making HTTP POST Requests with Httpx in Python

Author: Mohan Ganesan

Date: Feb 5, 2024

Httpx library in Python provides a modern and intuitive HTTP client for making POST requests to APIs and web services. It handles request headers, form data, timeouts, retries, and more.

Sending HTTP Requests in Python: Request vs Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Python applications often require HTTP requests. The request library is built-in, while requests is a more powerful third-party library that simplifies the process.

Sending and Receiving JSON Data with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Python Requests library makes it easy to send HTTP requests and receive responses in JSON format. It simplifies working with APIs and web services.

Rate Limiting Requests with aiohttp

Author: Mohan Ganesan

Date: Feb 22, 2024

Prevent abuse and reduce server load by rate limiting requests using aiohttp's ThrottleConcurrency middleware.

Python Requests Library: Making Authenticated POST Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

The Python Requests library provides a simple way to make HTTP requests in Python, including POST requests with Basic HTTP Authentication for authenticated API requests.

What is the full form of SOAP?

Author: Mohan Ganesan

Date: May 7, 2024

SOAP is an XML-based messaging protocol for web services that enables different applications to communicate over HTTP.

What is Requests Used For in Python?

Author: Mohan Ganesan

Date: Oct 22, 2023

Requests library simplifies working with HTTP APIs and web services in Python, including web scraping, API testing, interacting with web services, building web clients, fetching data, and automation.

Is REST API a code?

Author: Mohan Ganesan

Date: May 7, 2024

REST API is an architectural style for building web services, not a coding framework. It requires writing code for routing, controllers, models, etc.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!