Stories from the Web Crawling trenches in asynchronous programming

Making Concurrent Requests in Python: A Programmer's Guide

Author: Mohan Ganesan

Date: Nov 18, 2023

Handling multiple API calls and web scraping concurrently is critical for Python developers. This guide explores techniques for performant concurrent requests in Python.

Making Async HTTP Requests in Python with requests and asyncio

Author: Mohan Ganesan

Date: Feb 3, 2024

Python requests library provides API for HTTP requests. asyncio and aiohttp enable non-blocking requests. grequests uses asyncio for concurrent requests. asyncio is efficient for I/O heavy work.

Scraping Wikipedia Tables With Rust

Author: Mohan Ganesan

Date: Dec 6, 2023

Scraping Wikipedia allows for quick access to structured data, data availability, and hands-on practice with web scraping concepts. This article provides a step-by-step guide to scraping data on US presidents using web scraping techniques.

Building Asynchronous APIs with aiohttp and Queue

Author: Mohan Ganesan

Date: Mar 3, 2024

Asynchronous programming with aiohttp and queues in Python enables efficient web development and API creation.

Asyncio task exception handling

Author: Mohan Ganesan

Date: Mar 25, 2024

Asynchronous programming with asyncio in Python has advantages and challenges. Proper exception handling is key to creating robust asyncio code.

Implementing Scalable Async I/O with Python Asyncio Queues

Author: Mohan Ganesan

Date: Mar 25, 2024

Asyncio queues provide a great way to pass data between asynchronous tasks in Python. They enable building scalable asynchronous I/O flows without some of the downsides of threads or processes.

Understanding Asyncio Coroutines and Tasks in Python

Author: Mohan Ganesan

Date: Mar 17, 2024

Asynchronous programming in Python using coroutines and tasks. Coroutines define asynchronous behavior, while tasks actually run the coroutines and enable concurrency.

Is Python asynchronous or synchronous?

Author: Mohan Ganesan

Date: Mar 17, 2024

Python's asyncio module enables asynchronous I/O for improved concurrency. Use asyncio for I/O-bound tasks and when concurrency is needed.

异步HTTP客户端/服务器框架aiohttp入门指南

Author: Mohan Ganesan

Date: Mar 3, 2024

aiohttp is a powerful Python asynchronous network programming framework for building high-performance asynchronous IO applications.

Making Asynchronous HTTP Requests in Python

Author: Mohan Ganesan

Date: Feb 3, 2024

Python Requests library provides simple interface for making HTTP requests. Supports synchronous and asynchronous requests using threads or processes.

Async IO and Futures in Python: What's the Difference?

Author: Mohan Ganesan

Date: Mar 17, 2024

Asynchronous programming in Python with asyncio and futures. asyncio provides infrastructure for async I/O concurrency while futures represent eventual results of asynchronous operations.

Handling Errors Gracefully with Asyncio Exceptions

Author: Mohan Ganesan

Date: Mar 25, 2024

Asyncio provides an asynchronous programming framework in Python for non-blocking I/O code. Exception handling in asyncio requires special care, including handling CancelledError and propagating exceptions from tasks.

Async IO Sleep vs Time Sleep in Python - When to Use Each

Author: Mohan Ganesan

Date: Mar 17, 2024

When writing asynchronous Python code, use asyncio.sleep() for delays without blocking, and time.sleep() for pausing all processing in the current thread.

What is alternate to asyncio in Python?

Author: Mohan Ganesan

Date: Mar 17, 2024

Asyncio is Python's built-in asynchronous programming framework, but alternatives like Twisted and Trio are worth exploring.

What is the difference between parallel and async in Python?

Author: Mohan Ganesan

Date: Mar 24, 2024

Python offers two options for performing multiple tasks simultaneously: parallel programming, which leverages multiple CPU cores, and asynchronous programming, which allows long-running functions to yield control back while waiting.

Build High Performance Asyncio Web Servers in Python

Author: Mohan Ganesan

Date: Mar 25, 2024

Python's asyncio module allows you to write non-blocking, event-driven network code. This makes it possible to build very high performance web servers that can handle thousands of concurrent connections with very low resource usage.

Asyncio Concurrency in Python: Unlocking Asynchronous Magic

Author: Mohan Ganesan

Date: Mar 25, 2024

Concurrency is essential for building responsive and scalable applications. Asyncio in Python allows for asynchronous code, making the most of hardware resources.

What is the future in asyncio python ?

Author: Mohan Ganesan

Date: Mar 17, 2024

Asyncio enables asynchronous programming in Python. It is gaining popularity and offers performance improvements, new idioms, and integration with other languages. It is set to become an indispensable part of the Python ecosystem.

Async IO in Python with aiohttp

Author: Mohan Ganesan

Date: Mar 3, 2024

aiohttp brings the performance benefits of async I/O to Python web development while retaining a simple, Pythonic API.

What is the difference between asyncio and queue?

Author: Mohan Ganesan

Date: Mar 24, 2024

Asynchronous programming in Python with asyncio and queues. asyncio for I/O bound tasks, queues for CPU bound work. Different concurrency models and performance tradeoffs.

What is the difference between async and await?

Author: Mohan Ganesan

Date: Mar 24, 2024

Asynchronous programming in JavaScript can be achieved using async/await and promises. Async/await provides syntax that makes code easier to read and maintain, while promises lay the foundation for async/await.

Making Python Asynchronous: An Introduction to asyncio

Author: Mohan Ganesan

Date: Mar 17, 2024

Asynchronous programming in Python with asyncio allows for concurrent execution, improved speed and efficiency. It is useful for network programming and database access.

What is the difference between asyncio and synchronous?

Author: Mohan Ganesan

Date: Mar 24, 2024

Python includes both synchronous and asynchronous programming capabilities. Use synchronous code for simple scripts and CPU-bound processing. Use asyncio for I/O-bound work, parallel execution, and concurrency within a single thread.

What are the advantages of asyncio in Python?

Author: Mohan Ganesan

Date: Mar 17, 2024

Python's asyncio module opens up a whole new world of asynchronous programming, allowing code to execute concurrently and resulting in huge performance gains for I/O-bound applications.

What are the modes of asyncio python ?

Author: Mohan Ganesan

Date: Mar 17, 2024

Asynchronous programming in Python using asyncio module for building responsive and scalable applications.

Multithreading in Python: Choosing the Right Model

Author: Mohan Ganesan

Date: Mar 17, 2024

Multithreading in Python can improve performance and responsiveness. Choose the right model based on use case and tradeoffs. Options include threading, multiprocessing, and asyncio.

Tired of getting blocked while scraping the web?

ProxiesAPI handles headless browsers and rotates proxies for you.
Get access to 1,000 free API credits, no credit card required!