CODEWITHSUNDEEP
python-3.xseleniumxpathweb-scraping
Suresh Thiyam
Suresh Thiyam

Reputation: 1

How to select multiple rows values in xpath selenium using Python?

This is how my html code looks like of the table that i want to scrape

here is my full html code of the page that i want to scrape, check this one: https://pastebin.com/uMhqJmrf

This is my script in Python. It works for scraping only the first row, i want to scrape all the multiple rows that are shown in the table. This is how my table looks like that I want to scrape data. https://prnt.sc/vnrx2n.

import urllib
import requests
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select

try:
    options = Options()
    options.headless = True
    driver = Chrome('C:/Users/poison/Downloads/chromedriver_win32/chromedriver.exe', options=options)

    #driver = Chrome('C:/Users/poison/Downloads/chromedriver_win32/chromedriver.exe')
    driver.get("https://redacted.com/root/Default.aspx")

    username_form = driver.find_element_by_id("txtLoginID")
    password_form = driver.find_element_by_id("txtPassword")
    security_code_form = driver.find_element_by_id("txtCaptcha")

    captcha_image = driver.find_element_by_id("Image1Captcha")
    captcha_url = captcha_image.get_attribute("src")
    captcha_code = captcha_url.split("?Str=")[1]

    username_form.send_keys("redacted")
    password_form.send_keys("redacted")
    security_code_form.send_keys(captcha_code)

    driver.find_element_by_id("btnLogin").click()

    driver.get("https://redacted.com/root/Admin/Recharge_PendingRequest.aspx")

    driver.find_element_by_xpath('//td[./select[@name="ctl00$ContentPlaceHolder1$ddlServiceType"]]').click()
    driver.find_element_by_xpath('//div[@class="chosen-search"]/input').send_keys('dth\n').click()

    phone = driver.find_element_by_xpath('//tr[@class="RowStyle"]/td[4]').text
    amount = driver.find_element_by_xpath('//tr[@class="RowStyle"]/td[5]').text
    operator = driver.find_element_by_xpath('//tr[@class="RowStyle"]/td[6]').text

    # recharge_amount
    ResultText = "ID number: " + phone + "\n" + "Amount: " + amount + "\n" + "Operator: " + operator
    ParsedResultText = urllib.parse.quote_plus(ResultText)
    requests.get("https://api.telegram.org/bot1405ddddddsQL60wRzOCaSEi9WB5twPoP5Euw/sendMessage?chat_id=-10dddddd2091""&text={}".format(ParsedResultText))
    driver.quit()
except Exception:
    print("No pending Recharge Request!!!")

Upvotes: 0

Views: 1058

Answers (1)

DMart
DMart

Reputation: 2461

You're only selecting elements from one row.

Try something like this:

rows = driver.find_elements_by_xpath('//tr[@class="RowStyle"]') #get all the row elements
for row in rows: #loop through each row
    #pluck out the elements from each row
    phone = row.find_element_by_xpath('./td[4]').text
    amount = row.find_element_by_xpath('./td[5]').text
    operator = row.find_element_by_xpath('./td[6]').text
    print(phone, amount, operator)

Now you'll actually want to choose a different means of storing the values you select, but this gives you the jist,

Upvotes: 1

Related Questions