Stories from the Web Crawling trenches in Elixir

The Ultimate Floki Cheatsheet for Elixir

Author: Mohan Ganesan

Date: Oct 31, 2023

Floki makes it easy to parse and query HTML documents in Elixir using CSS selectors and tree traversal.

Making HTTP Requests Through a Proxy in Elixir with HTTPoison in 2024

Author: Mohan Ganesan

Date: Jan 9, 2024

Learn how to install HTTPoison in Elixir, make requests, configure global and per-request proxies, use SOCKS proxies, handle authentication and TLS, and manage IP blocks and captchas with proxy rotation services.

How to Build a Super Simple HTTP Proxy in Elixir in just 20 lines of code

Author: Mohan Ganesan

Date: Oct 1, 2023

Elixir makes it easy to build fast and scalable network applications. Here is a basic HTTP proxy server in less than 20 lines of Elixir code.

Scraping All Images from a Website with Elixir

Author: Mohan Ganesan

Date: Dec 13, 2023

Step-by-step guide to scraping a website for dog breed information and images using Elixir. Retrieve web page content, parse HTML, extract data, and download images.

Scraping Data from Wikipedia with Elixir

Author: Mohan Ganesan

Date: Dec 6, 2023

Learn how to scrape structured data from a Wikipedia table using Elixir. Use HTTPoison and Floki libraries to extract and transform data into a reusable format.

Web Scraping with Elixir & ChatGPT

Author: Mohan Ganesan

Date: Sep 25, 2023

Elixir is a great language for web scraping with ChatGPT. HTTPoison and Floki are useful libraries. ChatGPT provides explanations and code snippets. Proxies API is a robust solution for web scraping.

Scraping Multiple Pages in Elixir with HTTPoison and Floki

Author: Mohan Ganesan

Date: Oct 15, 2023

Web scraping in Elixir using HTTPoison and Floki libraries to extract data from multiple pages. Use proxies for scraping at scale.

Downloading Images from a Website with Elixir and Floki

Author: Mohan Ganesan

Date: Oct 15, 2023

Learn how to use Elixir and libraries like HTTPoison and Floki to download images from a Wikipedia page and extract data from HTML tables.

Web Scraping Yelp Business Listings using Elixir

Author: Mohan Ganesan

Date: Dec 6, 2023

Web scraping guide for extracting data from Yelp business listings using Elixir and Floki. Learn how to make HTTP requests, parse HTML, and extract information. Use premium proxies to bypass anti-bot measures.

Scraping Reddit Posts in Elixir

Author: Mohan Ganesan

Date: Jan 9, 2024

Web scraping tutorial using Elixir code to extract post information from Reddit. Learn how to install dependencies, make requests, parse HTML, and use CSS selectors.

Scraping New York Times News Headlines in Elixir

Author: Mohan Ganesan

Date: Dec 6, 2023

Learn how to use Elixir libraries like HTTPoison and Floki to automate web scraping and extract data from the New York Times homepage.

Scraping Real Estate Listings From Realtor in Elixir

Author: Mohan Ganesan

Date: Jan 9, 2024

Scrape real estate listings from Realtor.com for properties in San Francisco using Elixir code.

Scraping Booking.com Property Listings in Elixir in 2023

Author: Mohan Ganesan

Date: Oct 15, 2023

Learn how to scrape property listings from Booking.com using Elixir, HTTPoison, and Floki. Use proxies for scaling web scraping.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!