Do I need urllib3?

Feb 20, 2024 ยท 2 min read

Python provides several modules for making HTTP requests, with the main ones being the built-in urllib module and the popular third-party requests library. But there's another option that fills an important niche - urllib3. Here's when urllib3 can be useful.

urllib3 focuses specifically on the underlying HTTP connection - it handles all the socket level logic and TLS encryption. This allows it to provide useful capabilities like:

  • Connection pooling and reuse
  • File uploads with proper encoding
  • Thread safety and support for asynchronous requests
  • Automatic redirection
  • Proxy support
  • Here's a quick example of making a GET request with urllib3:

    import urllib3
    http = urllib3.PoolManager()
    r = http.request('GET', '')

    So when would you use urllib3 instead of urllib or requests?

  • If you need full control over the HTTP connection handling and want to manage things like redirection and retries yourself.
  • When making lots of requests in parallel - urllib3's connection pooling shines here.
  • In asynchronous applications - urllib3 supports async with aiohttp and Tornado.
  • If you want to wrap and extend the requests module - urllib3 is what requests is built on!
  • The main downsides are that urllib3 has a lower-level API and you lose some conveniences of requests like automatic JSON decoding. It also does not have authentication or session support built-in.

    So in summary, consider using urllib3:

  • If you want connection pooling and better performance for many requests.
  • When using asyncio or writing asynchronous code.
  • If you want to handle redirects, retries and TLS manually.
  • As a foundation for building on top of requests.
  • For most basic API clients, requests tends to be the best choice. But for large scraping or crawling projects or advanced use cases, take a look at urllib3!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl ""

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: