Stories from the Web Crawling trenches in HttpClient

Downloading Images from URLs in CSharp

Author: Mohan Ganesan

Date: May 5, 2024

Learn how to download images from URLs using C# with WebClient, HttpClient, RestSharp, Flurl, and System.Drawing.

Web Scraping with C++ & ChatGPT

Author: Mohan Ganesan

Date: Sep 25, 2023

C++ is a powerful language for web scraping with ChatGPT. Use libraries like libcurl and libxml2 for HTTP requests and HTML parsing. ChatGPT can provide explanations and generate code snippets. Get started now!

Scraping Reddit Posts in CSharp

Author: Mohan Ganesan

Date: Jan 9, 2024

Download and parse a Reddit page using AngleSharp in C# to extract information from posts.

Scraping All Images from a Website with Elixir

Author: Mohan Ganesan

Date: Dec 13, 2023

Step-by-step guide to scraping a website for dog breed information and images using Elixir. Retrieve web page content, parse HTML, extract data, and download images.

Web Scraping with CSharp & ChatGPT

Author: Mohan Ganesan

Date: Sep 25, 2023

Web scraping in C# using ChatGPT and HtmlAgilityPack for data extraction and code generation.

Scraping eBay Listings with Kotlin and HttpClient in 2023

Author: Mohan Ganesan

Date: Oct 5, 2023

eBay is a large online marketplace. This tutorial explains how to scrape and extract data from eBay listings using Kotlin and the HttpClient library.

Scraping Multiple Pages in Visual Basic with HtmlAgilityPack and HttpClient

Author: Mohan Ganesan

Date: Oct 15, 2023

Web scraping in Visual Basic using HtmlAgilityPack and HttpClient libraries to extract data from multiple pages. Use XPath queries and proxies for efficient data extraction.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!