Stories from the Web Crawling trenches in HtmlAgilityPack

Web Scraping with C++ & ChatGPT

Author: Mohan Ganesan

Date: Sep 25, 2023

C++ is a powerful language for web scraping with ChatGPT. Use libraries like libcurl and libxml2 for HTTP requests and HTML parsing. ChatGPT can provide explanations and generate code snippets. Get started now!

Web Scraping Wikipedia with CSharp

Author: Mohan Ganesan

Date: Dec 6, 2023

Learn how to scrape data from Wikipedia using C# and the HtmlAgilityPack library. Extract information from websites for data collection, analysis, and automation.

Downloading Images from a Website with CSharp and HtmlAgilityPack

Author: Mohan Ganesan

Date: Oct 15, 2023

Learn how to use C# and HtmlAgilityPack to download images from a Wikipedia page and extract data from HTML tables.

Scraping all the Images from a Website using CSharp

Author: Mohan Ganesan

Date: Dec 13, 2023

Learn how to scrape data and images from a website using C# and HtmlAgilityPack library. Extract data from a webpage, check HTTP status code, store data, and download images.

Downloading Images from a Website with VB and HtmlAgilityPack

Author: Mohan Ganesan

Date: Oct 15, 2023

Learn how to use Visual Basic and HtmlAgilityPack to download images from a Wikipedia page and extract data on dog breeds.

Web Scraping with CSharp & ChatGPT

Author: Mohan Ganesan

Date: Sep 25, 2023

Web scraping in C# using ChatGPT and HtmlAgilityPack for data extraction and code generation.

Scraping Craigslist Listings with CSharp

Author: Mohan Ganesan

Date: Oct 1, 2023

Learn how to scrape Craigslist apartment listings using C# and HtmlAgilityPack. Avoid IP blocking with a rotating proxy server.

Scraping Multiple Pages in CSharp with HtmlAgilityPack

Author: Mohan Ganesan

Date: Oct 15, 2023

Web scraping in C# using HtmlAgilityPack to extract data from multiple pages. Use proxies for scaling up and avoiding IP blocks.

Scraping Real Estate Listings from Realtor with CSharp

Author: Mohan Ganesan

Date: Jan 9, 2024

Scrape real estate listing data from Realtor.com using C# and HtmlAgilityPack library. Extract information like broker name, price, beds, baths, sqft, lot size, and address.

Scraping Multiple Pages in Visual Basic with HtmlAgilityPack and HttpClient

Author: Mohan Ganesan

Date: Oct 15, 2023

Web scraping in Visual Basic using HtmlAgilityPack and HttpClient libraries to extract data from multiple pages. Use XPath queries and proxies for efficient data extraction.

Scraping Craigslist Listings with Visual Basic

Author: Mohan Ganesan

Date: Oct 1, 2023

Learn how to scrape Craigslist apartment listings using Visual Basic and HtmlAgilityPack library. Avoid IP blocking with a rotating proxy server.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!