CODEWITHSUNDEEP
pythonselenium
Light
Light

Reputation: 31

Looping over Selenium WebElement

I am not getting the expected results from the following iteration. The issue is the iteration value is not updating in the browser.find_element_by_xpath function (4th line of the code below). It always yields the first result. Note that I get the expected results when I manually run the iteration by assigning sequentially values 1,2,3,4 to the variable review_num instead of using the argument + str(review_num)

    review_num = 0
    for review_num in range(216):   # for every review
        review_num = review_num + 1
        fixedelement = browser.find_element_by_xpath('//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div["+ str(review_num)"]')
        #fixedelement = browser.find_element_by_xpath('//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div[1]')
        #fixedelement = browser.find_element_by_xpath('//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div[2]')
        #fixedelement = browser.find_element_by_xpath('//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div[3]')
        #fixedelement = browser.find_element_by_xpath('//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div[4]')
        R_title = fixedelement.find_element_by_xpath('./h4')
        R_author = fixedelement.find_element_by_xpath('./div[2]/p[1]/span')
        R_stars = fixedelement.find_element_by_xpath('./div[1]/div[1]/div[1]/span')
        R_date = fixedelement.find_element_by_xpath('./div[1]/div[1]/div[2]/small')
        R_comment = fixedelement.find_element_by_xpath('./div[1]/div[3]')
        R_Yesvotes = fixedelement.find_element_by_xpath('./div[2]/div/div[1]/a[1]/span')
        R_Novotes = fixedelement.find_element_by_xpath('./div[2]/div/div[1]/a[2]/span')

        R_title_text = R_title.text
        R_author_text = R_author.text
        R_stars_text = R_stars.text
        R_date_text = R_date.text
        R_comment_text = R_comment.text
        R_Yesvotes_text = R_Yesvotes.text
        R_Novotes_text = R_Novotes.text
        print(R_author_text)
        with open(csvfile, "a", newline='', encoding='utf-8') as output:
            writer = csv.writer(output, dialect='excel')
            # writer.writerow(["namerow_id", "Name", "Position_Location"])
            writer.writerow([review_num, R_title_text, R_author_text, R_stars_text, R_date_text, R_comment_text, R_Yesvotes_text, R_Novotes_text])

Upvotes: 1

Views: 61

Answers (1)

wpercy
wpercy

Reputation: 10090

In the line

fixedelement = browser.find_element_by_xpath('//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div["+ str(review_num)"]')

you're searching for the element with xpath equal to 

'//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div["+ str(review_num)"]'

with the "str(review_num)" hardcoded into the xpath.

You want to do something like

xpath = '//*[@id="collapseReviews"]/div/div[2]/div[2]/div[1]/div[{}]'.format(review_num)
fixedelement = browser.find_element_by_xpath(xpath)

Upvotes: 2

Related Questions