CODEWITHSUNDEEP
python
Shikhar Gupta
Shikhar Gupta

Reputation: 47

Python script stops after a few minutes

I'm building a python script to check the price of an amazon item every 5-10 seconds. Problem is, the script stops 'working' after a few minutes. There is no output to the console but it shows up as 'running' in my processes.

I'm using requests sessions for making http requests and time to display the time of request.

My code is as follows;

target_price = raw_input('Enter target price: ')
url = raw_input('Enter the product url: ')
while True:
    delay=randint(5,10)

    print datetime.datetime.strftime(datetime.datetime.now(), '%Y-%m-%d %H:%M:%S')+': '+'Sleeping for ' + str(delay) + ' seconds'
    time.sleep(delay)
    try:
        with requests.Session() as s:
            page = s.get(url,headers=headers,proxies=proxyDict,verify=False,timeout=5)
            tree = html.fromstring(page.content)
            price = tree.xpath('//div[@class="a-row a-spacing-mini olpOffer"]/div[@class="a-column a-span2 olpPriceColumn"]/span[@class="a-size-large a-color-price olpOfferPrice a-text-bold"]/text()')[0]
            new_price = re.findall("[-+]?\d+[\.]?\d+[eE]?[-+]?\d*", price)[0]
            old_price = new_price
            print new_price
            if float(new_price)<float(target_price):
                print 'Lower price found!'
                mydriver = webdriver.Chrome()
                send_simple_message()
                login(mydriver)
                print 'Old Price: ' + old_price
                print 'New Price: ' + new_price
            else:
                print 'Trying again'
    except Exception as e:
        print e
        print 'Error!'

EDIT: I've removed the wait() function and used time.sleep instead.

EDIT2: When I use Keyboard interrupt to stop the script, here's the output

    Traceback (most recent call last):
  File "checker.py", line 85, in <module>
    page = s.get(url,headers=headers,proxies=proxyDict,verify=False,timeout=5)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 488, in get
return self.request('GET', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 423, in send
timeout=timeout
  File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 589, in urlopen
self._prepare_proxy(conn)
  File "C:\Python27\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 797, in _prepare_proxy
conn.connect()
  File "C:\Python27\lib\site-packages\requests\packages\urllib3\connection.py",
line 267, in connect
self._tunnel()
  File "C:\Python27\lib\httplib.py", line 729, in _tunnel
line = response.fp.readline()
KeyboardInterrupt

Is it requests that is running into an infinite loop?

Upvotes: 3

Views: 2825

Answers (1)

afsafzal
afsafzal

Reputation: 602

The timeout argument to the s.get() function is tricky. Here I found a good explanation for its unusual behavior. The timeout will stop the process if the requested url does not respond but it wouldn't stop it if it respond infinitely.

In your case, the connection is established nut the requested page is just sending responses in an infinite loop.

You can set a timeout for the whole function call: Timeout function if it takes too long to finish

Upvotes: 1

Related Questions