urllib certificate verify failed

Feb 6, 2024 ยท 2 min read

When using urllib in Python to make HTTPS requests, you may encounter SSL certificate verification errors like "certificate verify failed". This means urllib is unable to verify the authenticity of the server's SSL certificate. Here are some things you can try to resolve this:

Check for Expired Certificates

Check if the server is using an expired or invalid SSL certificate. urllib will fail to verify expired certificates. The server needs to renew its SSL certificate.

Disable Certificate Verification

You can disable certificate verification in urllib by setting context=ssl._create_unverified_context(). However this is unsafe and should only be done for testing:

import ssl
import urllib.request 

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

urllib.request.urlopen(url, context=ctx)

Update Certificates

Make sure urllib has the latest SSL certificates needed to verify the server's certificate chain. On Linux, update the ca-certificates package. On Mac, update your Keychain certificates.

Use Certificate Pinning

For more security, pin the server's certificate or public key. urllib will only allow that pinned certificate from the server:

import ssl
import urllib.request

ctx = ssl.create_default_context(cafile="server-cert.pem") 
urllib.request.urlopen(url, context=ctx)

The key is properly configuring SSL contexts in urllib for your specific environment. Verify if the server has certificate issues first before tweaking the client. Disabling verify should only be temporary to isolate the issue. Consider certificate pinning for production use cases.

Browse by tags:

Browse by language:

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!