Stories from the Web Crawling trenches in selectors

The Ultimate Select.rs Cheat Sheet for Rust

Author: Mohan Ganesan

Date: Oct 31, 2023

select.rs is a robust HTML/XML scraping library for Rust. This cheat sheet covers its features, including installation, loading documents, selecting nodes, traversing nodes, extracting/modifying nodes, creating/inserting/removing nodes, output formats, caching and persistence, headless browsers, validation, encoding, advanced selectors, caching and performance, common recipes, troubleshooting, and ecosystem libraries.

The Ultimate Cheerio Web Scraping Cheat Sheet

Author: Mohan Ganesan

Date: Oct 31, 2023

Cheerio is a fast, flexible web scraping library for Node.js. This cheat sheet provides a comprehensive reference of its syntax and capabilities.

Web Scraping with Rust & ChatGPT

Author: Mohan Ganesan

Date: Sep 25, 2023

Rust is a great language for web scraping with ChatGPT's help. It involves sending HTTP requests, extracting data, and using selectors. ChatGPT can provide explanations and generate code snippets. A web scraping API like Proxies API can be used for more robust solutions.

Scraping Multiple Pages in Rust with reqwest and selectors

Author: Mohan Ganesan

Date: Oct 15, 2023

Web scraping in Rust using reqwest and selectors crates to extract data from multiple pages. Use proxies for scaling up scraping.

How to Build a Reddit Scraper in Java

Author: Mohan Ganesan

Date: Jan 9, 2024

Learn how to scrape Reddit posts using Java, web scraping, HTML parsing, selectors, and user-agent headers.

Scraping Reddit Posts in Kotlin

Author: Mohan Ganesan

Date: Jan 9, 2024

Scrape Reddit posts using Kotlin script, send HTTP requests, parse HTML, and extract key data using selectors.

Scraping Craigslist Listings with Rust

Author: Mohan Ganesan

Date: Oct 1, 2023

Learn how to scrape Craigslist apartment listings using Rust and the reqwest and selectors crates.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!