Scraping eBay Listings with Visual Basic and HtmlDocument in 2023

Oct 5, 2023 ยท 3 min read

eBay is one of the largest online marketplaces with millions of active listings at any given time. In this tutorial, we'll walk through how to scrape and extract key data from eBay listings using Visual Basic and the HtmlDocument library.

Setup

We'll need to add a reference to System.Xml.dll to work with HTML documents.

We'll also define the starting eBay URL and a string for the user agent header:

Imports System.Xml

Module eBayScraper

Dim url As String = "<https://www.ebay.com/sch/i.html?_nkw=baseball>"

Dim userAgent As String = "Mozilla/5.0..."

Replace the user agent with your browser's user agent string.

Fetch the Listings Page

We can use the WebClient class to download the HTML:

Dim webClient As New WebClient()
webClient.Headers.Add("User-Agent", userAgent)

Dim html As String = webClient.DownloadString(url)

Dim htmlDoc As New HtmlDocument()
htmlDoc.LoadHtml(html)

The user agent header is added to the request. We load the HTML into an HtmlDocument.

Extract Listing Data

We can use XPath syntax to find the listings divs:

Dim listings = htmlDoc.DocumentElement.SelectNodes("//div[contains(@class,'s-item__info')]")

For Each listing In listings

  Dim title As String = listing.SelectSingleNode(".//div[@class='s-item__title']").InnerText
  Dim url As String = listing.SelectSingleNode(".//a[@class='s-item__link']").GetAttributeValue("href", "")
  Dim price As String = listing.SelectSingleNode(".//span[@class='s-item__price']").InnerText

  ' Get other fields like seller, shipping, etc

Next

We extract the text and attributes from the selected elements.

Print Results

We can print the extracted info:

Console.WriteLine("Title: " & title)
Console.WriteLine("URL: " & url)
Console.WriteLine("Price: " & price)

Console.WriteLine(New String("=", 50)) ' Separator

This will output each listing's data.

Full Code

Here is the full code to scrape eBay listings:

Imports System.Xml

Module eBayScraper

Dim url As String = "<https://www.ebay.com/sch/i.html?_nkw=baseball>"

Dim userAgent As String = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36"

Dim webClient As New WebClient()
webClient.Headers.Add("User-Agent", userAgent)

Dim html As String = webClient.DownloadString(url)

Dim htmlDoc As New HtmlDocument()
htmlDoc.LoadHtml(html)

Dim listings = htmlDoc.DocumentElement.SelectNodes("//div[contains(@class,'s-item__info')]")

For Each listing In listings

  Dim title As String = listing.SelectSingleNode(".//div[@class='s-item__title']").InnerText
  Dim url As String = listing.SelectSingleNode(".//a[@class='s-item__link']").GetAttributeValue("href", "")
  Dim price As String = listing.SelectSingleNode(".//span[@class='s-item__price']").InnerText

  Dim details As String = listing.SelectSingleNode(".//div[@class='s-item__subtitle']").InnerText
  Dim sellerInfo As String = listing.SelectSingleNode(".//span[@class='s-item__seller-info-text']").InnerText
  Dim shippingCost As String = listing.SelectSingleNode(".//span[@class='s-item__shipping']").InnerText
  Dim location As String = listing.SelectSingleNode(".//span[@class='s-item__location']").InnerText
  Dim sold As String = listing.SelectSingleNode(".//span[@class='s-item__quantity-sold']").InnerText

  Console.WriteLine("Title: " & title)
  Console.WriteLine("URL: " & url)
  Console.WriteLine("Price: " & price)
  Console.WriteLine("Details: " & details)
  Console.WriteLine("Seller: " & sellerInfo)
  Console.WriteLine("Shipping: " & shippingCost)
  Console.WriteLine("Location: " & location)
  Console.WriteLine("Sold: " & sold)

  Console.WriteLine(New String("=", 50))

Next

End Module

Browse by tags:

Browse by language:

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!