Reputation:
This below code gets data from a table in a web page.
After crawling through a page it goes to next page and does the same thing again. The URL of the page doesn't change while moving to next page.
I want to use a loop so that it goes on for 50 or 75 times and breaks.
driver.get(site)
mytable = driver.find_element_by_css_selector('.table.table...nline')
for row in mytable.find_elements_by_css_selector('tr'):
for cell in row.find_elements_by_tag_name('td'):
sys.stdout=open("abcd.txt","a+")
print(cell.text)
sys.stdout.close()
driver.find_element_by_xpath("//li[@class='button next']/a").click()
I have tried using while loop, but i'm getting issues while appending the file.
Upvotes: 2
Views: 63
Reputation: 71610
Try slicing:
driver.get(site)
mytable = driver.find_element_by_css_selector('.table.table...nline')
for row in mytable.find_elements_by_css_selector('tr')[:50]:
for cell in row.find_elements_by_tag_name('td'):
sys.stdout=open("abcd.txt","a+")
print(cell.text)
sys.stdout.close()
driver.find_element_by_xpath("//li[@class='button next']/a").click()
Upvotes: 1