CODEWITHSUNDEEP
pythonlistpython-2.7webweb-scraping
user3387184
user3387184

Reputation: 13

iterate the list in python

I have a loop inside loop i'm using try n catch once get error try n catch works fine but loop continues to next value. What I need is that where the loop breaks start from the same value don't continue to next so how i can do that with my code [like in other languages: in c++, it is i--]

for 
r = urllib2.urlopen(url) 
       encoding = r.info().getparam('charset')
       html = r.read()
c = td.find('a')['href']
urls = []
urls.append(c)
#collecting urls from first page then from those url collecting further info in below loop

    for abc in urls:
       try:
           r = urllib2.urlopen(abc) 
           encoding = r.info().getparam('charset')
           html = r.read()
       except Exception as e:
           last_error = e
           time.sleep(retry_timeout) #here is the problem once get error then switch from next value

I need a more pythonic way to do this. Waiting for a reply. Thank you.

Upvotes: 0

Views: 84

Answers (1)

Luc DUZAN
Luc DUZAN

Reputation: 1319

Unfortunatly, there is no simple way to go back with iterator in Python :

http://docs.python.org/2/library/stdtypes.html

You should be interested in this stackoverflow's thread : Making a python iterator go backwards?

For your particular case, i will use a simple while loop :

url = []
i = 0
while i < len(url): #url is list contain all urls which contain infinite as url updates every day
    data = url[i]
    try:
        #getting data from there
        i+=1
    except:
        #shows the error received and continue to next loop i need to make the loop start from same position

The problem with the way, you want to handle your problem is that you will risk to go on a infinite loop. For example if a link is broken r = urllib2.urlopen(abc) will always run an exception and you will always stay at the same position. You should consider doing something like that :

r = urllib2.urlopen(url) 
encoding = r.info().getparam('charset')
html = r.read()
c = td.find('a')['href']
urls = []
urls.append(c)
#collecting urls from first page then from those url collecting further info in below loop
NUM_TRY = 3
for abc in urls:
   for _ in range(NUM_TRY):
       try:
           r = urllib2.urlopen(abc) 
           encoding = r.info().getparam('charset')
           html = r.read()
           break #if we arrive to this line, it means no error occur so we don't need to retry again
                #this is why we break the inner loop
       except Exception as e:
           last_error = e
           time.sleep(retry_timeout) #here is the problem once get error then switch from next value

Upvotes: 3

Related Questions