When using urllib in Python to make HTTPS requests, you may encounter SSL certificate verification errors like "certificate verify failed". This means urllib is unable to verify the authenticity of the server's SSL certificate. Here are some things you can try to resolve this:
Check for Expired Certificates
Check if the server is using an expired or invalid SSL certificate. urllib will fail to verify expired certificates. The server needs to renew its SSL certificate.
Disable Certificate Verification
You can disable certificate verification in urllib by setting
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
Make sure urllib has the latest SSL certificates needed to verify the server's certificate chain. On Linux, update the ca-certificates package. On Mac, update your Keychain certificates.
Use Certificate Pinning
For more security, pin the server's certificate or public key. urllib will only allow that pinned certificate from the server:
ctx = ssl.create_default_context(cafile="server-cert.pem")
The key is properly configuring SSL contexts in urllib for your specific environment. Verify if the server has certificate issues first before tweaking the client. Disabling verify should only be temporary to isolate the issue. Consider certificate pinning for production use cases.