How to Clear the Cache in Python Requests

Feb 1, 2024 ยท 2 min read

When making HTTP requests with the Python Requests library, it utilizes caching for performance. This saves time by avoiding repeat network requests for the same resource. However, sometimes you need to clear the cache to fetch fresh data or troubleshoot issues. Here are a few ways to clear the Requests cache in Python.

Use session.close()

The easiest way is closing the current session. This clears all cached connections and cookies:

import requests

session = requests.Session()
response = session.get('https://example.com') 

# Clear cache
session.close()  

Creating a new Session instance also works.

Manually Clear the Cache

You can manually clear the cache too by setting the cache attribute on a Session to None:

import requests

session = requests.Session()

# Clear cache
session.cache = None

This clears the cache while reusing the same Session.

Use Cache Control Headers

Another approach is using the "Cache-Control" request header to avoid caching:

import requests

url = 'https://api.example.com/data'

headers = {'Cache-Control': 'no-cache'}

response = requests.get(url, headers=headers)

The "no-cache" directive tells web servers and proxies to fetch fresh data.

Things to consider

  • Clearing the cache for every request causes overhead. Balance performance vs freshness requirements.
  • Server responses may still get cached by proxies and browsers. The above clears the Python Requests cache only.
  • Browse by language:

    Tired of getting blocked while scraping the web?

    ProxiesAPI handles headless browsers and rotates proxies for you.
    Get access to 1,000 free API credits, no credit card required!