Is Python asynchronous or synchronous?

Mar 17, 2024 ยท 4 min read

Python is generally considered a synchronous programming language - when you execute a line of code, the next line waits for the first to finish before executing. However, Python does provide asynchronous capabilities through asyncio and other libraries. So when is async useful and how can you leverage it in Python?

The Difference Between Asynchronous and Synchronous

To understand asyncio, let's quickly define synchronous vs asynchronous:

  • Synchronous - Code executes linearly from top to bottom. Each statement waits for the previous one to finish before executing. This can "block" execution when waiting for I/O.
  • Asynchronous - Code executes non-linearly. I/O operations are handled in the background so that execution can continue without blocking. This enables concurrent operations.
  • # Synchronous
    import time
    print('Start')
    time.sleep(2) 
    print('End')
    
    # Asynchronous
    import asyncio
    
    async def main():
        print('Start')
        await asyncio.sleep(2)  
        print('End')
    
    asyncio.run(main())

    So Python itself runs synchronously, but asyncio provides infrastructure to write asynchronous code.

    When to Use Asynchronous Programming

    The main benefit of async is that while one part of the program waits for I/O (like network requests), other parts can continue executing. This enables concurrency - multiple things happening at the same time.

    Some examples where async programming shines:

  • I/O-intensive tasks - network requests, reading/writing files
  • Servers handling many concurrent connections
  • UI/UX interactions that shouldn't "block"
  • However, async introduces overhead and complexity. It's best to start simple and only utilize when:

  • Performance bottlenecks confirmed to be caused by synchronous I/O
  • Concurrency specifically needed
  • Async IO - asyncio

    The asyncio module is Python's official library for async IO. Some key concepts:

    Coroutines - Functions that can pause execution without blocking the thread. Defined with async def:

    async def fetch_data():
       print('fetching!')
       await asyncio.sleep(2)
       print('done!')

    Event loop - Runs and schedules coroutines. asyncio.run() creates a loop:

    asyncio.run(fetch_data())

    Awaitable - Objects that can be awaited on within coroutines, like delays or I/O operations.

    We await on awaitables to pause coroutines instead of blocking:

    await asyncio.sleep(2)
    data = await fetch_url('example.com') 

    Asyncio handles switching between coroutines transparently as they execute and await.

    Concurrency Patterns with Asyncio

    Here are some common patterns that leverage asyncio concurrency:

    Parallelize Independent Tasks

    Run unrelated coroutines concurrently:

    async def fetch_url(url):
       # fetch and return response
    
    urls = ['url1', 'url2', 'url3']
    
    async def main():
       tasks = []
       for url in urls:
          task = asyncio.create_task(fetch_url(url))  
          tasks.append(task)
       
       await asyncio.gather(*tasks)
    
    asyncio.run(main()) 

    Producer/Consumer Queues

    A queue lets one coroutine produce data for another to consume asynchronously:

    queue = asyncio.Queue()
    
    async def producer():
       for i in range(5):
          await queue.put(i)
       await queue.put(None) # Signal we're done
    
    async def consumer():
       while True:
          item = await queue.get()
          if item is None:
             break
          print(f'Consumed {item}')
    
    asyncio.run(asyncio.gather(producer(), consumer()))

    This enables concurrent data processing pipelines.

    Async Context Managers

    Asyncio provides async versions of common context managers like files and locks:

    async with aiofiles.open('file.txt') as f:
       contents = await f.read()

    The async context handles opening/closing the file without blocking.

    Going Further with Asyncio

    Here are some tips to leverage asyncio further:

  • Use asyncio.create_task() to launch background coroutines
  • Create reusable concurrency helpers with asyncio.AbstractEventLoop
  • Integrate async code with async frameworks like aiohttp, databases, etc
  • Profile performance to compare async vs sync options
  • And some things to watch out for:

  • Blocking the event loop negates the benefits of async
  • Nesting many levels of awaits may hurt performance
  • Async code can get complex quickly - validate need first
  • In Summary

    While Python executes synchronously by default, the asyncio module enables asynchronous I/O for improved concurrency:

  • Use asyncio for I/O-bound tasks and when concurrency needed
  • Coroutines + event loop handle async execution
  • Parallel tasks, queues, context managers to leverage concurrency
  • Profile code to confirm async helps before refactoring
  • Asyncio does take some re-thinking for asynchronous code design. But when used properly, it enables efficient concurrent programs in Python.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: