Encoding URLs with urllib quote

Feb 6, 2024 · 2 min read

When building URLs in Python, it is often necessary to encode special characters to ensure the URL is valid. The urllib.parse.quote() function provides an easy way to accomplish this.

Let's look at a quick example:

from urllib.parse import quote

url = "https://www.example.com/search?term=café"
print(quote(url))

This would output:

https%3A%2F%2Fwww.example.com%2Fsearch%3Fterm%3Dcaf%C3%A9

As you can see, the accented character é has been converted to its percent encoded form %C3%A9. This ensures the URL can be processed correctly.

The quote() function handles encoding all reserved characters defined in RFC 3986. This includes:

  • Non-alphanumeric characters like spaces, braces, asterisks etc.
  • Non-ASCII or accented letters like é or ß
  • Special symbols like ® or €
  • Without proper encoding, these characters could cause issues when used in URLs.

    For example, spaces would get interpreted as a separator rather than part of the term:

    https://www.example.com/search?term=café au lait

    Would become:

    https%3A%2F%2Fwww.example.com%2Fsearch%3Fterm%3Dcaf%C3%A9%20au%20lait

    See how the space got converted to %20?

    The quote() function takes care of all of this automatically. You just pass in the full URL string and it will encode the special characters.

    One thing to note is that quote() will not encode slashes or the question mark in the URL. Those are considered safe and valid characters for URLs.

    In summary, urllib.parse.quote() is extremely useful when constructing URLs in Python. It handles all the intricacies of percent encoding special characters. Be sure to use it if building URLs with non-ASCII text or other symbols that could cause issues. Encoding the parameters ensures that URLs will be processed correctly on the server side.

    Browse by tags:

    Browse by language:

    Tired of getting blocked while scraping the web?

    ProxiesAPI handles headless browsers and rotates proxies for you.
    Get access to 1,000 free API credits, no credit card required!