CODEWITHSUNDEEP
pythonselenium
Sai
Sai

Reputation: 49

How to run multiple Selenium tests back to back

I'm trying to perform dynamic web scraping by using Selenium and Python. The website I am trying to scrape has an input where almost 600 or so locations can be chosen. Each location has different facilities.

I want to scrape the name of the facilities as well as the address for each location.

I have the list of locations in an excel file. I used Pandas to import those locations and then tried to do a for loop where I go through each location, enter the location in the input, and then scrape the facilities that are outputted underneath.

But my for loop hasn't worked as intended and so every time I run the script, Selenium driver opens up and I scrape the facilities as intended for the first location. But then nothing happens after that. My goal is for the script to continue, either new Selenium browsers opening or a new location is entered into the input to scrape the facilities of the second location. 

df = pd.read_excel('MA Cities - Licensed Child Care Search.xlsx')

for i in df: 
    driver = webdriver.Chrome(options = options)
    driver.get('https://eeclead.force.com/EEC_ChildCareSearch')
    res = driver.execute_script("return document.documentElement.outerHTML")

    search = driver.find_element_by_id('j_id0:j_id2:j_id29:city')
    search.send_keys(i)
    search.send_keys(Keys.ENTER)

Solved it by putting the driver part in a function and then calling that function with a for loop from 0 to # of cities I am dealing with.

Upvotes: 1

Views: 126

Answers (1)

Sai
Sai

Reputation: 49

Solved it by putting the selenium driver part in a function and then calling that function with a for loop from 0 to # of cities I am dealing with.

for i in range(x): 
    my_function(i)

And then in the function added 

search.send_keys(df.iloc[i])

Upvotes: 0

Related Questions