Stories from the Web Crawling trenches in HTML content

How to Select Elements by Text in XPath

Author: Mohan Ganesan

Date: Jan 9, 2024

XPath is used for navigating XML and HTML documents in web scraping. It can select elements based on text content using contains function or exact match.

Rendering HTML Responses with aiohttp

Author: Mohan Ganesan

Date: Mar 3, 2024

aiohttp provides flexible options for returning HTML to clients, from raw strings to rendered templates to streaming output.

Returning HTML Responses with aiohttp in Python

Author: Mohan Ganesan

Date: Mar 3, 2024

aiohttp allows easy return of HTML content in Python web applications and APIs. Use template engines and response streaming for robust web apps.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!