CODEWITHSUNDEEP
pythonseleniumfor-loopxpath
Adrian Reichert
Adrian Reichert

Reputation: 55

How to insert a variable in xpath within a for loop?

for i in range(length):
        # print(i)
        driver.execute_script("window.history.go(-1)")
        range = driver.find_element_by_xpath("(//a[@class = 'button'])[i]").click()


        content2 = driver.page_source.encode('utf-8').strip()
        soup2 = BeautifulSoup(content2,"html.parser")
        name2 = soup2.find('h1', {'data-qa-target': 'ProviderDisplayName'}).text
        phone2 = soup2.find('a', {'class': 'click-to-call-button-secondary hg-track mobile-click-to-call'}).text
        print(name2, phone2)

Hey guy I am trying to scrape the First and last Name, Telephone for each person this website: https://www.healthgrades.com/family-marriage-counseling-directory. I want the (l.4) button to adapt to the variable (i). if i manually change i to a number everything works perfectly fine. But as soon as I placed in the variable i it doesn't work, any help much appreciated!

Upvotes: 0

Views: 510

Answers (1)

cruisepandey
cruisepandey

Reputation: 29362

Instead of this :

range = driver.find_element_by_xpath("(//a[@class = 'button'])[i]").click()

do this :

range = driver.find_element_by_xpath(f"(//a[@class = 'button'])[{i}]").click()

Update 1 :

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(50)
driver.get("https://www.healthgrades.com/family-marriage-counseling-directory")

for name in driver.find_elements(By.CSS_SELECTOR, "a[data-qa-target='provider-details-provider-name']"):
  print(name.text)

Output :

Noe Gutierrez, MSW
Melissa Huston, LCSW
Gina Kane, LMHC
Dr. Mary Marino, PHD
Emili-Erin Puente, MED
Richard Vogel, LMFT
Lynn Bednarz, LCPC
Nicole Palow, LMHC
Dennis Hart, LPCC
Dr. Robert Meeks, PHD
Jody Davis
Dr. Kim Logan, PHD
Artemis Paschalis, LMHC
Mark Webb, LMFT
Deirdre Holland, LCSW-R
John Paul Dilorenzo, LMHC
Joseph Hayes, LPC
Dr. Maylin Batista, PHD
Ella Gray, LCPC
Cynthia Mack-Ernsdorff, MA
Dr. Edward Muldrow, PHD
Rachel Sievers, LMFT
Dr. Lisa Burton, PHD
Ami Owen, LMFT
Sharon Lorber, LCSW
Heather Rowley, LCMHC
Dr. Bonnie Bryant, PHD
Marilyn Pearlman, LCSW
Charles Washam, BCD
Dr. Liliana Wolf, PHD
Christy Kobe, LCSW
Dana Paine, LPCC
Scott Kohner, LCSW
Elizabeth Krzewski, LMHC
Luisa Contreras, LMFT
Dr. Joel Nunez, PHD
Susanne Sacco, LISW
Lauren Reminger, MA
Thomas Recher, AUD
Kristi Smith, LCSW
Kecia West, LPC
Gregory Douglas, MED
Gina Smith, LCPC
Anne Causey, LPC
Dr. David Greenfield, PHD
Olga Rothschild, LMHC
Dr. Susan Levin, PHD
Ferguson Jennifer, LMHC
Marci Ober, LMFT
Christopher Checke, LMHC

Process finished with exit code 0

Update 2 :

leng = len(driver.find_elements(By.CSS_SELECTOR, "a[data-qa-target='provider-details-provider-name']"))
for i in range(leng):
  driver.find_element_by_xpath(f"(//a[text()='View Profile'])[{i}]").click()

Upvotes: 1

Related Questions