CODEWITHSUNDEEP
pythonpython-requests
Shadow
Shadow

Reputation: 9427

Is there a way to globally override requests' timeout setting?

We have usages of the requests library littered throughout our project. Recently we came across a bug in one of our destinations where it froze mid transaction, and decided to just hold the connection open.

Naturally, our application followed suit.

Is there a environment variable, or some other way to set the timeout? Even if it's significant (say, 30 seconds) it should be enough to stop the entire works from stopping because of one service. If possible, it should be global so that I don't have to find every single use, and so that people can't forget to add it in the future.

Upvotes: 45

Views: 25283

Answers (6)

Filip Malczak
Filip Malczak

Reputation: 3194

Unfortunately, looking at the code, there is no possibility to set a global default value. I was kinda surprised by that, as I would expect that to be quite common use case. If you start a feature request, please let me know (e.g. in comments to this post).

The reason for that is that methods like get(...), post(...), etc are all just thin wrappers over Session.request(...) method (requests.get(...) creates new one-shot session, just for a single request). That method takes timeout as argument, and does not inspect Session internals for a value if there is no timeout argument, so you always have to put it there manually, like 2ps proposed in his answer.

Sources:

Revised on master on 31.08.2020 as well as on 12.08.2024. Line numbers have changed, but methods stayed the same. The answer stays the same.

PS. See this pull request. Disclaimer: it's mine.

Upvotes: 18

Pratyush
Pratyush

Reputation: 5488

The simplest way is to "shim" the session's request function:

import requests
import functools

s = requests.Session()
s.request = functools.partial(s.request, timeout=3)

# now all get, post, head etc requests should timeout after 3 seconds
# following will fail
s.get('https://httpbin.org/delay/6')

# we can still pass higher timeout when needed
# following will succeed
s.get('https://httpbin.org/delay/6', timeout=7)

Upvotes: 58

codeskyblue
codeskyblue

Reputation: 428

Instead you could inherit the requests.Session class and rewrite request function, like this.

HTTP_TIMEOUT = 30

class TimeoutRequestsSession(requests.Session):
    def request(self, *args, **kwargs):
        kwargs.setdefault('timeout', HTTP_TIMEOUT)
        return super(TimeoutRequestsSession, self).request(*args, **kwargs)

session = TimeoutRequestsSession()
session.get('https://www.google.com') # connection timeout is default 30s

Upvotes: 17

vicusbass
vicusbass

Reputation: 1804

Why not?:

    for method in ("get", "options", "head", "post", "put", "patch", "delete"):
        setattr(
            session,
            method,
            functools.partial(getattr(session, method), timeout=timeout),
        )

Upvotes: 2

Roel Kramer
Roel Kramer

Reputation: 391

Two ways to make this happen. It involves some dirty monkey patching. Tested on python 3.6.

Put this somewhere in a file like main.py, an init.py or urls.py and make sure it is called.

Option 1: patch the requests method

import requests

def request_patch(slf, *args, **kwargs):
    print("Fix called")
    timeout = kwargs.pop('timeout', 2)
    return slf.request_orig(*args, **kwargs, timeout=timeout)

setattr(requests.sessions.Session, 'request_orig', requests.sessions.Session.request)
requests.sessions.Session.request = request_patch

Option 2: Patch the Session class

import requests

class SessionTimeoutFix(requests.Session):

    def request(self, *args, **kwargs):
        print("Fix called")
        timeout = kwargs.pop('timeout', 2)
        return super().request(*args, **kwargs, timeout=timeout)

requests.sessions.Session = SessionTimeoutFix

Upvotes: 10

2ps
2ps

Reputation: 15926

You can certainly do it per call with the timeout parameter:

requests.get('http://www.google.com', timeout=10)

The timeout parameter specifies the number of seconds or a tuple for (connection timeout, read timout). You can read more about it here.

If you want to make this global, the easiest way is to refactor all of the calls into a wrapper class and make sure that everyone is using the wrapper class to call this API endpoint. I did a quick check of the requests code and did not see the use of a global override for timeout since the default behavior is to wait for data.

Upvotes: 0

Related Questions