Stories from the Web Crawling trenches in SSL certificate

Accessing HTTPS Sites with Self-Signed Certs in Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

Methods to securely access HTTPS sites using self-signed certificates with Python Requests: certifi bundle, custom PEM certs, REQUESTS_CA_BUNDLE, SSLContext.

urllib certificate verify failed

Author: Mohan Ganesan

Date: Feb 6, 2024

urllib in Python may encounter SSL certificate verification errors. Try checking for expired certificates, disabling certificate verification, updating certificates, and using certificate pinning.

Troubleshooting SSL Certificate Errors with Python Requests

Author: Mohan Ganesan

Date: Feb 3, 2024

When using Python Requests library for HTTPS requests, you may encounter SSL certificate errors. Try updating OS, specifying custom CA bundle, or disabling certificate verification.

Using AFNetworking Proxies for Web Scraping in 2024

Author: Mohan Ganesan

Date: Jan 9, 2024

Setting up a basic AFNetworking proxy, working with different proxy protocols, advanced proxy functionality, troubleshooting common AFNetworking proxy problems.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!