Stories from the Web Crawling trenches in nest_asyncio

Making Asynchronous Code Synchronous in aiohttp

Author: Mohan Ganesan

Date: Feb 22, 2024

The aiohttp library in Python allows for asynchronous HTTP requests. This article covers techniques to integrate aiohttp with synchronous code or external libraries, including using run_in_executor(), asyncio.to_thread(), running an event loop in a thread, and the nest_asyncio decorator.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!