Stories from the Web Crawling trenches in UnicodeDecodeErrors

Fixing aiohttp UnicodeDecodeErrors

Author: Mohan Ganesan

Date: Mar 3, 2024

Fixing UnicodeDecodeErrors in aiohttp: specify encoding, check actual encoding, decode manually, re-encode text

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!