CODEWITHSUNDEEP
paginationweb-scrapingkimono
Florent
Florent

Reputation: 1

kimono pagination error with Amazon

I am trying to scrape Amazon result pages using Kimono. For instance I type "Sony" on Amazon.co.uk search engine and want to scrape the results to get product names. I am able to do it for one page but I'd like to scrape multiple pages using pagination feature. So far I can't get the pagination to work. I select "Next Page" as the link for pagination but then I get a E20 error.

Anyone knows what I'm doing wrong? Could it be the structure of Amazon pages which prevents the use of pagination feature?

Thanks! Flo

Upvotes: 0

Views: 219

Answers (1)

Igor Savinkin
Igor Savinkin

Reputation: 6267

Anyone knows what I'm doing wrong?

Seems you've encountered business directories tough case. It's almost always scrape-proof walled/pitched. Amazon with its billions of items for sure keeps high walls against data theft and such a light scraping tool as the kimono scraper for sure (99,99%) will stumble over it. See the tips and tricks on scraping business directories.

Quote

Those kind of sites are especially vigilant about using anti-scraping measures to avoid data leaks. So be ready for unexpected pits falls and unbreakable firewalls...

My suggestion is you to pick any off-the-shelf scraping software, plug proxy service into it and go ahead scraping.

Upvotes: 0

Related Questions