CODEWITHSUNDEEP
python-3.xlistloopsseleniumfor-loop
Bisbot
Bisbot

Reputation: 127

Print text output into single list with Selenium web scraper for loop

I am running the following program which scrapes this website. The program uses a list which fills 3 search fields on the website then prints the text of the selected page. It does this over and over again until the list_2.txt comes to an end.

Here is the code: 

list_2 = [['7711564', '14', '93'], ['0511442', '7', '27']]

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep


driver = webdriver.Firefox()
driver.get("https://www.airdrie.ca/index.cfm?serviceID=284")

for query in list_2:

   driver.find_element_by_name("whichPlan").send_keys(query[0])
   driver.find_element_by_name("whichBlock").send_keys(query[1])
   driver.find_element_by_name("whichLot").send_keys(query[2])

   driver.find_element_by_name("legalSubmit").click()
   sleep(3)

   text_element = driver.find_elements_by_xpath("//div[@class='datagrid']")
   text_element2 =
   driver.find_elements_by_xpath("//table[@class='quickkey_tbl ']")

   txt = [x.text for x in text_element]
   print(txt, '\n')
   txt2 = [x.text for x in text_element2]
   print(txt2, '\n')

   driver.back()
   driver.refresh()
   sleep(2)

I want to be able to print ALL the results from each loop/iteration into a single list. I tried using += but this ends up printing double outputs for the first item on my list only.

Upvotes: 0

Views: 218

Answers (1)

Ratmir Asanov
Ratmir Asanov

Reputation: 6459

You can try something like this:

results_list = []

for query in list_2:
   ...

   txt = [x.text for x in text_element]
   print(txt, '\n')
   txt2 = [x.text for x in text_element2]
   print(txt2, '\n')
   results_list.append(txt + txt2)
   ...

Hope it helps you!

Upvotes: 2

Related Questions