Why is multithreading not faster in python?

Mar 24, 2024 ยท 2 min read

Python's multithreading capabilities are often misunderstood when it comes to speeding up code execution. While multithreading allows multiple threads to run seemingly in parallel, it doesn't actually allow Python code to run faster on a single CPU core due to something called the Global Interpreter Lock (GIL).

What is the Global Interpreter Lock (GIL)?

The GIL is a mutex in Python that allows only one thread to execute Python bytecodes at a time even in a multi-threaded application. This prevents multiple threads from running Python code at once and causing problems that can occur with shared memory spaces.

So what does this mean? It means that Python threads don't actually run parallel code at the same time - they run one at a time through the GIL. This serialization limits the performance gains from multithreading in CPU-bound Python programs.

When Does Multithreading Help?

Multithreading can still provide performance benefits for I/O-bound applications, or applications that spend a lot of time waiting for external resources like network requests or file I/O. While one thread waits for I/O to complete, another thread can run useful Python code.

For example:

# Thread 1 does I/O
data = download_file() 

# Thread 2 runs useful Python code while waiting

So in I/O-bound cases, multithreading allows you to maximize utilization of resources that would otherwise be wasted waiting.

Tips for Multithreading in Python

Some tips:

  • Use multithreading for I/O-bound tasks rather than CPU-bound tasks
  • Consider multiprocessing for CPU-bound tasks
  • Use pools of workers rather than hundreds of threads
  • Avoid shared memory between threads when possible
  • So in summary - while Python's multithreading capabilities are useful, they come with some caveats due to the GIL. Understanding those caveats allows you to properly leverage threads for better performance where applicable.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />


    Don't leave just yet!

    Enter your email below to claim your free API key: