How to write URL in Python?

Feb 8, 2024 ยท 2 min read

When working with web applications, APIs, or scraping websites in Python, you'll inevitably need to handle URLs. Here are some best practices for writing, encoding, parsing, and modifying URLs in Python.

Writing URLs

You can specify URLs in Python using normal strings like:

url = "" 

Prepend http:// or https:// for absolute URLs. You can also construct URLs programmatically by concatenating strings:

root_url = ""
path = "/products"
url = root_url + path

Encoding URLs

When including URLs in queries or path segments, ensure special characters are URL encoded using the urlencode() function from urllib:

from urllib.parse import urlencode
params = {"q": "green eggs & ham"} 
print(urlencode(params)) # q=green+eggs+%26+ham

This converts characters like &, =, and spaces to percent-encoded characters like %26.

Parsing URL Components

To manipulate a URL's protocol, hostname, path etc individually, use urllib's urlparse() method:

from urllib.parse import urlparse

url = ""
parsed = urlparse(url)

print(parsed.scheme) # 'http' 
print(parsed.hostname) # ''

You can modify components of the ParseResult and rebuild with urlunparse().

Practical Usage

When making HTTP requests with the requests module, it will automatically handle correct URL encoding and parsing for you. But when scraping websites or handling URLs for an API, utilize Python's URL handling utilities like urlencode() and urlparse() to ensure your URLs are valid.

The key is using the right tool for the job - Python makes handling URLs easy, as long as you leverage the built-in libraries.

Browse by tags:

Browse by language:

The easiest way to do Web Scraping

Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

Try ProxiesAPI for free

curl ""

<!doctype html>
    <title>Example Domain</title>
    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />


Don't leave just yet!

Enter your email below to claim your free API key: