Log all requests from the python-requests module

I am using python Requests. I need to debug some OAuth activity, and for that I would like it to log all requests being performed. I could get this information with ngrep, but unfortunately it is not possible to grep https connections (which are needed for OAuth)

How can I activate logging of all URLs (+ parameters) that Requests is accessing?

The underlying urllib3 library logs all new connections and URLs with the logging module, but not POST bodies. For GET requests this should be enough:

    import logging

    logging.basicConfig(level=logging.DEBUG)

which gives you the most verbose logging option; see the logging HOWTO for more details on how to configure logging levels and destinations.

Short demo:

    >>> import requests
    >>> import logging
    >>> logging.basicConfig(level=logging.DEBUG)
    >>> r = requests.get('http://httpbin.org/get?foo=bar&baz=python')
    INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): httpbin.org
    DEBUG:requests.packages.urllib3.connectionpool:"GET /get?foo=bar&baz=python HTTP/1.1" 200 353

The following messages are logged:

  • INFO: New connections (HTTP or HTTPS)
  • INFO: Dropped connections
  • INFO: Redirects
  • WARN: Connection pool full (if this happens often increase the connection pool size)
  • WARN: Retrying the connection
  • DEBUG: Connection details: method, path, HTTP version, status code and response length

From: stackoverflow.com/q/16337511

Back to homepage or read more recommendations: