CODEWITHSUNDEEP
pythonpandasseleniumloopsfor-loop
user12285770
user12285770

Reputation:

IndexError: list index out of range in python while something not found

How to Fix IndexError: list index out of range

I'm Doing Scraping But if My Script Can't Find Something It Give This Error

IndexError: list index out of range

I Want To Continue to next link not break but my script break and not go with second url

Here is my python code:

import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

plus = "+ "

with open("Sans Fransico.csv","r") as s:
    s.read()

df = pd.read_csv('Yelp+Scraping_Sans+Fransico.csv') # Get all the urls from the excel
mylist = df['Urls'].tolist() #urls is the column name

driver = webdriver.Chrome()
for url in mylist:

    driver.get(url)
    wevsite_link = driver.find_elements_by_css_selector(".text--offscreen__373c0__1SeFX+ .link-size--default__373c0__1skgq")
    phone = driver.find_elements_by_css_selector(".text--offscreen__373c0__1SeFX+ .text-align--left__373c0__2pnx_")



    items = len(wevsite_link)
    with open("Sans Fransico.csv", 'a',encoding="utf-8") as s:
        for i in range(items):
            if wevsite_link[i].text == '':
                s.write(phone[i].text + "\n")
            if [i] == '':
                s.write('N' + "," + 'N' + "\n")
                s.write('N' + "," + 'N' + "\n")
            if wevsite_link[i].text == '' and phone[i].text == '':
                s.write('' + "," + '' + "\n")
            else:
                s.write(phone[i].text + "," + wevsite_link[i].text + "\n")

driver.close()
print ("Done")

ERROR:

Traceback (most recent call last):
  File ".\seleniuminform.py", line 36, in <module>
    s.write(phone[i].text + "," + wevsite_link[i].text + "\n")
IndexError: list index out of range

Upvotes: 1

Views: 447

Answers (2)

Guy
Guy

Reputation: 50899

Missing items are not empty string, they don't exist. You can use itertools.zip_longest to iterate over both lists

with open("Sans Fransico.csv", 'a',encoding="utf-8") as s:
    for combination in itertools.zip_longest(wevsite_link, phone):
        s.write(f'{combination[0].text if combination[0] else "N"}, {combination[1].text if combination[1] else "N"}\n')

Upvotes: 2

tst
tst

Reputation: 381

if the errors are expected, you can wrap your main loop in a try/except´:

try:
   for url in mylist:
        .....

except Exception as e:
   print(e)

That will let you keep going and still give you info about where errors occured.

Upvotes: 2

Related Questions