Stories from the Web Crawling trenches in extraction

Loading HTML Files into BeautifulSoup for Web Scraping

Author: Mohan Ganesan

Date: Oct 6, 2023

BeautifulSoup makes it straightforward to load HTML for parsing and extraction. Use Python's built-in html.parser or choose others like lxml or html5lib. Selenium may be needed for dynamic pages.

The Ultimate Gumbo C++ Cheatsheet

Author: Mohan Ganesan

Date: Oct 31, 2023

Gumbo is an HTML5 parsing library in C++ that allows for easy manipulation and extraction of HTML. It provides various functions for selecting, traversing, and manipulating nodes in the DOM.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!