CODEWITHSUNDEEP
pythonseleniumweb-scrapingscrapydata-mining
Renuka
Renuka

Reputation: 7

How to click on "next" button using selenium or scrapy in python

I was trying to scrape some data from flipkart.com using scrapy. I got everything other than going to the next page. first I tried using scrapy followed by selenium. actually, a class has 2links for the previous page and for the next page.

with scrapy: I am unable to fetch any result. I am getting the blank output.

with selenium: whenever I try to click on the next page it works fine from page 1 to page 2. but then instead of going on the 3rd page, it's coming back to the second page.

is there any other way to do this? how should I distinguish the URLs for the previous page and the next page?

Upvotes: 1

Views: 272

Answers (1)

amey
amey

Reputation: 11

just append [-1] so that driver only fetches the last occurring result for the given class/id for flipkart the code is as follows

driver.find_elements_by_xpath('//a[@class="_1LKTO3"]')[-1].click()

Upvotes: 1

Related Questions