Authenticating Requests Through a Proxy with Digest Auth in Python

Feb 3, 2024 ยท 2 min read

When sending requests through a proxy server that requires authentication, you'll need to configure the Python Requests module to handle the proxy and digest authentication. Digest auth is more secure than basic auth, so proxies often use it to protect access.

Here's how to setup Requests to pass credentials and data through an authenticated proxy.

Configure the Proxy and Auth

First configure the proxy URL and authentication credentials to use:

proxies = {
    "http": "http://user:password@10.10.1.10:3128/",
    "https": "http://user:password@10.10.1.10:1080/",
}

We provide the proxy URLs including the username and password embedded in the standard URL format.

Next create a dict with the authentication realm and credentials:

auth = HTTPDigestAuth("realm","username","password")

The realm will be provided by the proxy, and we'll pass the related username and password.

Make Authenticated Requests

With the proxy and authentication setup, we can now make requests:

import requests

response = requests.get("https://api.example.com", proxies=proxies, auth=auth)
print(response.status_code)

The proxies dict and auth object are passed to requests.get(). This will route the request through the proxy and handle sending the digest authentication headers.

Requests will automatically generate the authentication nonce values and re-attempt requests if it receives a 401 unauthorized response.

Troubleshooting Issues

If you receive connection errors or 401 unauthorized responses, check a few things:

  • Validate the proxy URLs and credentials are correct
  • Check if the proxy allows requests to the destination URLs
  • Try toggling the proxy type between HTTP and HTTPS
  • Consider handling the authentication manually by generating nonce and response hash values
  • Getting authenticated requests working through proxies involves some trial and error, but this approach covers the main configuration needed in Python.

    Browse by tags:

    Browse by language:

    Tired of getting blocked while scraping the web?

    ProxiesAPI handles headless browsers and rotates proxies for you.
    Get access to 1,000 free API credits, no credit card required!