CODEWITHSUNDEEP
pythonpython-2.7asynchronous
haimen
haimen

Reputation: 2015

How to cancel a request after 5 minutes without a reply

I have the following function,

import requests

def get_url_type(data):
    x = {}
    for i in range(0,len(data)):
        print i        
        try:
            x[i] = requests.head(data['url'][i]).headers.get('content-type')
        except:
            x[i] = 'Not Available'
    return(x)

This function returns the URL type of each URL that is being passed to it and whenever there is no response, it throws error which is caught using exception. My problem here is, some of the requests take more than 5-10 mins time which is too much on production environment. I want the function to return "Not Available" when it takes more than 5 mins. When I did a research about it, it was mentioned to convert the function to asynchronous one. I have trying to change it without much success.

The following is what I have tried,

import asyncio  
import time  
from datetime import datetime

async def custom_sleep():  
    print('SLEEP', datetime.now())
    time.sleep(5)

My objective is, whenever the request function takes more than 5 mins, it should return "Not available" and move to the next iteration.

Can anybody help me in doing this?

Thanks in advance !

Upvotes: 2

Views: 2438

Answers (1)

Glenn D.J.
Glenn D.J.

Reputation: 1965

It seems you just want a request to time out after a given time has passed without reply and move on to the next request. For this functionality there is a timeout parameter you can add to your request. The documentation on this: http://docs.python-requests.org/en/master/user/quickstart/#timeouts.

With a 300 seconds (5 minutes) timeout your code becomes:

requests.head(data['url'][i], timeout=300)

The asynchronous functionality you are mentioning has actually a different objective. It would allow your code to not have to wait the 5 minutes at all before continuing execution but I believe that would be a different question.

Upvotes: 2

Related Questions