How to fix SSLError in Python requests

Oct 22, 2023 · 7 min read

Introduction

The requests module in Python provides an easy interface for making HTTP requests. However, when making requests to HTTPS sites, you may encounter SSLError exceptions. Properly handling SSL certificates is crucial for secure communication, so understanding and resolving these errors is important.

In this comprehensive guide, we’ll dig into the common SSLError messages in Python requests, their underlying causes, and effective ways to address them.

What is SSLError in Python Requests?

SSLError is a general exception for SSL-related errors that occur when making HTTPS requests using the requests module.

Some common subclasses of SSLError you may encounter are:

  • SSLCertVerificationError - Raised when certificate verification fails
  • SSLZeroReturnError - Signifies an SSL connection has been closed
  • SSLWantReadError - Socket needs more data to complete handshake
  • These errors mean your Python code is unable to securely connect to the remote server due to an issue with SSL certificates.

    SSL verification is critical because it validates the server's identity and encrypts traffic to prevent snooping. So while disabling verification may seem like an easy fix, it compromises security and should be avoided in production.

    When Does SSLError Occur in Python Requests?

    Some common triggers for SSLError in requests are:

    Expired or Invalid Server Certificate

    If the HTTPS server uses an expired or invalid certificate, verification will fail with an SSL error. This is easy to diagnose by inspecting the certificate.

    Hostname Mismatch

    If the request URL hostname doesn't match the Common Name or Subject Alternative Names on the certificate, you'll get a hostname mismatch error.

    For example, requesting https://www.example.com with a certificate for example.com will trigger this.

    Incomplete Certificate Chain

    The server's certificate must chain up to a trusted root CA that the client recognizes. If intermediate certificates are missing, the chain is broken, causing SSL errors.

    Outdated CA Bundle

    Python requests relies on the certifi module for root CA certificates to verify against. If certifi is outdated on your system, certificates may fail to validate, triggering SSLErrors.

    Old SSL/TLS Version on Server

    If the HTTPS server uses outdated SSL/TLS versions like SSLv2 or SSLv3, the client may refuse to connect, throwing a SSL handshake error.

    Modern security standards require at least TLS v1.2 or higher.

    How to Fix SSLError in Python Requests

    Now that we know what causes SSLErrors, let's go over some effective resolution techniques:

    Disabling SSL Certificate Verification

    You can disable SSL verification by passing verify=False to requests:

    requests.get("<https://expired.badssl.com/>", verify=False)
    

    This will suppress any SSL errors, but it's insecure and should never be used in production.

    Configuring CA Certificates for Verification

    Rather than disabling verification completely, it's better to properly configure certificate authorities for validation:

    # Use certifi CA bundle (default)
    requests.get("<https://example.com>", verify=certifi.where())
    
    # Use system-wide CA bundle
    import pip._vendor.requests as requests
    requests.get("<https://example.com>")
    
    # Provide custom CA bundle
    requests.get("<https://example.com>", verify="/path/to/ca-bundle.pem")
    

    You can also update certifi to the latest version for more up-to-date root certificates:

    pip install -U certifi
    

    Generating and Providing a Client Certificate

    For servers requiring client certificate authentication, you need to explicitly provide a certificate and key:

    requests.get("<https://example.com>", cert=("client.crt", "client.key"))
    

    This will perform mutual authentication and prevent man-in-the-middle attacks.

    Updating Server's SSL Configuration

    Sometimes SSL errors originate from the server side. Things to check include:

  • Server is using a modern TLS version like 1.2 or 1.3
  • Certificate is valid and properly chained to a trusted root CA
  • Private key permissions are correct
  • OpenSSL is up-to-date if applicable
  • Debugging Common SSLError Messages

    Finally, let's go over some frequent SSLError messages and how to diagnose them:

    CERTIFICATE_VERIFY_FAILED - Indicates the client could not validate the server's certificate against known CAs. Inspect the certificate chain and CA bundle.

    hostname 'example.com' doesn't match 'wronghost' - Hostname mismatch between request URL and certificate CN/SANs. Double check the hostname spelling.

    bad signature - Means the certificate signature is invalid. Probably an issue with the cert itself.

    tlsv1 alert protocol version - Suggests the server is using old SSLv1 which clients reject. Upgrade server to TLS 1.2 or higher.

    sslv3 alert handshake failure - Similarly indicates server uses outdated SSLv3. Upgrade protocol version on server side.

    Using SSLContext for More Control

    The SSLContext class allows you to finely customize SSL behavior in requests:

    import ssl
    
    context = ssl.create_default_context(cafile="custom_ca.pem")
    context.load_cert_chain("client.crt", "client.key")
    
    requests.get("<https://example.com>", ssl_context=context)
    

    This gives you control over certificate verification, protocol versions, ciphers, and more.

    Conclusion

    I hope this guide gives you a comprehensive understanding of the common SSLErrors faced in Python requests, their underlying causes, and helpful fixes for them.

    Key takeaways are:

  • Properly resolve errors instead of disabling verification
  • Update CA bundles, fix certificates, and use TLS 1.2+
  • Use SSLContext for full control over SSL behavior
  • With robust certificate handling, you can avoid frustrating SSLErrors and securely access HTTPS resources using Python requests!

    FAQ

    Why am I getting SSLError in Python requests?

    An SSLError in Python requests usually means there is an issue with the SSL certificate verification. This could happen if the certificate is invalid, expired, or the hostname in the certificate doesn't match the URL you are requesting.

    import requests
    requests.get('<https://expired-cert.badssl.com/>')
    # SSLError
    

    How do I fix SSL certificate verification failed error in Python requests?

    You can disable SSL certificate verification in requests by setting verify=False. However, this is insecure and not recommended. A better option is to provide the path to the correct CA bundle or directory using verify.

    requests.get('<https://url>', verify='/path/to/certbundle')
    

    How to disable SSL certificate verification in Python requests?

    requests.get('<https://url>', verify=False)
    

    Disabling verification is insecure. Use proper certificates instead.

    What causes SSLError in Python requests?

    Common causes are expired certificates, wrong hostnames in the cert, intermediary certificate issues, or using self-signed certificates without proper verification.

    How to handle SSLError in Python requests?

  • Check the certificate details and validate it manually
  • Disable verification as a last resort (not recommended)
  • Provide path to correct CA bundle for verification
  • Add certificate to request if using self-signed cert
  • How to add SSL certificate verification in Python requests?

    requests.get('<https://url>', cert=('client.crt', 'client.key'))
    

    Pass the certificate and key as a tuple to cert.

    How does Python requests handle SSL certificates?

    Requests verifies SSL certificates automatically using the certifi root CA bundle. You can override this by providing your own via the verify parameter.

    How to check SSL certificate validity in Python requests?

    import ssl
    import socket
    ctx = ssl.create_default_context()
    s = ctx.wrap_socket(socket.socket(), server_hostname='hostname')
    s.connect(('url', 443))
    cert = s.getpeercert()
    # validate cert
    

    Manually check the certificate returned by getpeercert().

    How to ignore SSL certificate errors in Python requests?

    requests.get('https://', verify=False)
    

    Not recommended, but disables SSL verification and ignores errors.

    What is the proper way to use SSL certificates with Python requests?

    Pass the path to the correct CA bundle via verify and optionally provide your client certificate and key via cert. This handles SSL verification properly without disabling security.

    Browse by tags:

    Browse by language:

    Tired of getting blocked while scraping the web?

    ProxiesAPI handles headless browsers and rotates proxies for you.
    Get access to 1,000 free API credits, no credit card required!