Uploading Files in Python Requests: A Guide

Feb 3, 2024 ยท 3 min read

Sending file uploads via HTTP requests is a common task in many Python applications. This guide covers how to upload files using the requests library and multipart/form-data.

Overview

The requests library makes it simple to send HTTP requests in Python. To upload files, we need to send a POST request with the file data formatted as multipart/form-data.

Here's what we'll cover:

  • What is multipart/form-data and why it's needed for file uploads
  • Constructing a multipart/form-data request with requests
  • Uploading files, images, and other binary data
  • Setting custom filenames
  • Dealing with redirects and responses
  • Handling exceptions and troubleshooting
  • Follow along with the examples below to learn how to integrate file uploading into your Python applications.

    Why Multipart/Form-Data for File Uploads

    Browser file upload forms use the multipart/form-data content type to send file data to the server. Unlike regular form posts, multipart encodes file data separately allowing large binary payloads to be transferred efficiently.

    The requests library properly handles multipart encoding under the hood. We just need to construct the request correctly.

    Uploading a File with Requests

    Uploading a file is just a matter of attaching the binary data as well as metadata to identify the file contents.

    Here's an example of posting an image to an upload endpoint:

    import requests
    
    url = 'https://api.example.com/upload'
    files = {'file': open('image.png', 'rb')}
    
    r = requests.post(url, files=files)

    Breaking this down:

  • files is a dictionary with the key being the form field name
  • The value is a file-like object opened for reading bytes ('rb')
  • requests handles reading the file contents and multipart encoding
  • The filename is derived from the file object name, but can be overridden
  • The server receives the uploaded filename, content type, and binary file data for further processing.

    Uploading Multiple Files

    Simply add additional key/value pairs to the files dictionary:

    files = {
        'file1': open('photo.jpg', 'rb'), 
        'file2': open('document.pdf', 'rb')
    }
    
    r = requests.post(url, files=files)

    Each file is transferred with a unique form field name for the server to distinguish.

    Setting Custom Filenames

    By default the filename sent to the server is based on the file object name. Pass a tuple with the desired filename instead:

    files = {
        'file': ('myphoto.jpg', open('photo.jpg', 'rb'))
    }

    Now the uploaded filename sent to the server is myphoto.jpg while the actual file on disk is still photo.jpg.

    Handling the Response

    As with all requests calls, validate the response after uploading:

    if r.status_code == requests.codes.ok:
        print('Upload successful')
    else:
        print('Error, got response:', r.status_code)

    Check the status code to see if the upload succeeded. Additionally, parse the response body for server-provided metadata.

    Exceptions and Troubleshooting

    If the upload fails, requests may raise exceptions like ConnectionError or Timeout:

    try:
        r = requests.post(url, files=files, timeout=10)
    except requests.exceptions.Timeout:
        print('Request timed out') 
    except requests.exceptions.RequestException as req_err:
        print('Error:', req_err)

    Catch general RequestException or specific exceptions like Timeout to handle errors.

    Additionally, enable verbose logging and inspect request parameters to troubleshoot odd server behaviors.

    Summary

    That covers the main techniques for uploading files via the excellent requests library! The key points are:

  • Use multipart/form-data for efficient file transfer
  • Attach file-like objects to the files parameter
  • Set custom filenames if needed
  • Validate responses and handle exceptions/errors
  • With these basics, you can integrate robust file upload capabilities into APIs, web scrapers, and automation scripts built in Python.

    Browse by tags:

    Browse by language:

    Tired of getting blocked while scraping the web?

    ProxiesAPI handles headless browsers and rotates proxies for you.
    Get access to 1,000 free API credits, no credit card required!