Reputation: 2329
I am dealing with very heavy load from Googlebot, I have hundreds of sites running on multiple IPs all on the same server and googlebot is constantly crawling these sites causing a lot of availability issues to those sites.
I've checked the IPs and they seem to be legit (66.249.66.XX
) so it doesn't look like its other bots spoofing, but i see weird things like google using negative numbers for paging, for example ?page=-274
. I've even had to add code to check if the value is < 0
to use 1
instead to avoid problems.
Any idea why is google doing things like that and how to prevent it?
Upvotes: 0
Views: 25