Stories from the Web Crawling trenches in Axios

The Complete Guide to Retrying Failed Requests with Axios

Author: Mohan Ganesan

Date: Jan 9, 2024

Automated retries using Axios interceptors provide reliability, speed, scalability, and resilience. Configuring retries involves setting the number of retries, delay between retries, and conditional retries. The Axios-Retry plugin simplifies the process. Other libraries like retry-axios offer similar capabilities. Testing and debugging retry logic is important, and caution must be taken to avoid circular retries. Axios is widely used in React apps and can be used for any HTTP backend. Automated retries are essential for building robust apps that handle remote services.

Web Scraping All The Images From a Website in Node.js

Author: Mohan Ganesan

Date: Dec 13, 2023

Automate data collection from websites using web scraping with Node.js, axios, and cheerio. Extract dog breed information and images from a Wikipedia page.

Scraping Wikipedia Pages with Node.js

Author: Mohan Ganesan

Date: Dec 6, 2023

Scrape Wikipedia using Node.js with axios and cheerio to extract structured data for various use cases.

Using Proxies with Axios in 2024

Author: Mohan Ganesan

Date: Jan 9, 2024

Learn how to integrate proxies with Axios for efficient web scraping and bot development. Avoid IP bans and scale your projects with ease.

Using Proxies in Axios in Node.js for Web Scraping in 2024

Author: Mohan Ganesan

Date: Jan 9, 2024

Configure proxies for Node.js web scraping using Axios library. Learn about proxy options, authentication, rotating proxies, environment variables, custom logic, and proxy services like Proxies API.

Scraping Real Estate Listings From Realtor in Node.js

Author: Mohan Ganesan

Date: Jan 9, 2024

Code to extract real estate listing data from Realtor.com for properties in San Francisco using Axios and Cheerio.

Scraping Yelp Business Listings in NodeJS

Author: Mohan Ganesan

Date: Dec 6, 2023

Learn how to scrape business listings from Yelp using web scraping techniques and premium proxies with Node.js and Axios.

How to call API URL?

Author: Mohan Ganesan

Date: May 7, 2024

When building an application, you'll often need to retrieve or send data to an API. This guide covers the basics of making API requests in your code.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!