CODEWITHSUNDEEP
python
user2351394
user2351394

Reputation: 93

Scraping Google Results with Python

Is there a way to scrape more than 64 results from google with python without getting my IP address instantly blocked?

Upvotes: 2

Views: 335

Answers (2)

darioguarascio
darioguarascio

Reputation: 1087

64 result is the limit? Sounds weird to me!

Even with browser, I can navigate till the 100th page with no problem.

I'm very curious about how did you reach this limit.

Anyway: classical possible solutions are:

  • proxying ( IE: tor )
  • delaying requests
  • randomly switch user agent

Upvotes: 1

Frederick Roth
Frederick Roth

Reputation: 2830

I use tsocks and ssh-tunnels to machines with other ip addresses to achieve this.

Upvotes: 1

Related Questions