CODEWITHSUNDEEP
pythonrequesturllib
qvws
qvws

Reputation: 1

Retry Request When Its Failed

I Programmed this code to get website content but there is a problem when connection is filed the program stop and dont try to reconnect

            url= 'https://website.com'
            def get_page_content(url, head):
              """
              Function to get the page content
              """
              req = Request(url, headers=head)
              return urlopen(req)
            head = {
              'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36',
              'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
              'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.3',
              'Accept-Encoding': 'none',
              'Accept-Language': 'en-US,en;q=0.8',
              'Connection': 'keep-alive',
              'refere': 'https://example.com',
              'cookie': """your cookie value ( you can get that from your web page) """
            }

            data = get_page_content(url, head).read()

I searched in the internet but didn't find solution.

Upvotes: -3

Views: 850

Answers (2)

Brandon Pardi
Brandon Pardi

Reputation: 211

Python has try, except blocks you can add. Basically you put the code that's failing (specifically the lines that throws errors) into the try section, and then the except portion will contain code that you want to do in the event of an error. You can use this with a flag in a loop to keep trying to connect a certain number of times

success_flag = False
while success_flag == False: # keep looping until success
     try: # code in here will attempt to run
          req = Request(url, headers=head)
          page = urlopen(req)
          success_flag = True # mark as successful run if got through the code that may fail
          return page
     except Exception as e: # this code will run in the event of a failure
          print(e) # just print the exception before trying again. 
          # possibly change something before reconnecting depending on error thrown

keep in mind you don't have the traceback posted, so there may be an issue with the code itself. This code will just do as you asked and that is keep trying again. You may need to change something before trying a reconnect

Upvotes: 0

david valencia
david valencia

Reputation: 109

You could use the status to retry when the requests fail.

get_page_content(url, head) has the property status. If the status is 200, 201, or 204 everything is ok. In any other case you should handle it.

Upvotes: 0

Related Questions