Debugging HTTP Requests in Python with Request Logging

Feb 3, 2024 ยท 2 min read

When working with the Python requests library to make HTTP requests, it can be invaluable to log the full request and response to debug issues. Here are some simple techniques to add comprehensive logging to your Python requests scripts.

Log Entire Request and Response to Console

The quickest way to log requests and responses is to use the built-in logging in requests. Add this before making the request:

import logging
import requests 

This will print the full URL, headers, and body of both requests and responses to the console.

Log to File for Production

For apps in production, log to a file instead of console:

import logging
logging.basicConfig(filename='app.log', level=logging.DEBUG)

This will append the logs to app.log for later debugging.

Log Specific Parts of Requests/Responses

To log specific parts like URL, status code, headers, or body, use the Python logging library:

import requests
import logging

logger = logging.getLogger('requests_logger')

r = requests.get('')
logger.debug(f'URL: {r.url}')
logger.debug(f'Status Code: {r.status_code}')
logger.debug(f'Headers: {r.headers}')
logger.debug(f'Body: {r.json()}')

This allows logging exactly what you need from requests without getting overwhelmed by too much data.


Adding logging to Python requests provides visibility into issues when making HTTP requests. Use the built-in stderr logging for quick debugging, log to a file for production, and log specific attributes like URL, status and headers for targeted debugging. Proper request/response logging can save hours of time troubleshooting Python apps.

Browse by tags:

Browse by language:

The easiest way to do Web Scraping

Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you

Try ProxiesAPI for free

curl ""

<!doctype html>
    <title>Example Domain</title>
    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />


Don't leave just yet!

Enter your email below to claim your free API key: