Making Scheme-Agnostic HTTP Requests in Python

Feb 3, 2024 ยท 2 min read

When making HTTP requests in Python using the popular requests library, the scheme (http:// or https://) is usually specified in the URL. However, sometimes you may want to make a request in a scheme-agnostic way without hardcoding http or https.

This can be useful in situations where:

  • You want to support both HTTP and HTTPS connections depending on some runtime configuration
  • You are making requests to internal services that use custom schemes
  • You want to simplify your code by not having to specify the scheme repeatedly
  • Fortunately, the requests library makes this easy to do.

    Constructing Scheme-Agnostic URLs

    Instead of hardcoding the scheme like:

    url = ""

    You can omit it:

    url = "//"

    This constructs a scheme-relative URL that will use the default scheme determined by the request context at runtime.

    Making Requests

    Once you have your scheme-agnostic URL, making the actual request works the same:

    import requests
    url = "//"
    response = requests.get(url, timeout=5)

    The request will default to either HTTP or HTTPS based on factors like if the host URL supports HTTPS, environment variables, etc.

    Wrap Up

    Constructing scheme-relative URLs and allowing requests to handle the scheme selection simplifies your code and enables greater flexibility. Give it a try next time you want your Python application to support both HTTP and HTTPS connections!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl ""

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: