CODEWITHSUNDEEP
google-app-engine
user2085368
user2085368

Reputation: 725

Being flooded with traffic, and the remote IP reported in app engine logs as starting with 10

I host a web service, and have recently getting many HTTP requests (up to several thousand per second) from IP addresses that start with 10 according to the attached log. From my limited networking knowledge, this prefix means the IP is a local one, not a WAN IP. Why would app engine report traffic from Google's own LAN IP? Furthermore, because of this it seems I'm unable to blacklist that IP range, which has been costing me quite a lot in quota fees! Any ideas of why I'd be seeing local IP addresses in the logs for these requests, and how i can block them before they reach my application?

IP address starting with 10.

Upvotes: 2

Views: 112

Answers (1)

user2085368
user2085368

Reputation: 725

Sigh, embarrassingly, there is an obvious reason a Google App Engine application would see an IP with prefix 10 - It's Google's Crawler. The issue was I was generating too many unique URLs and it was trying to crawl them all, leading to the obscene traffic volumes I was seeing. So I was, in a way, DOS attacking myself by letting the crawler know about too many unique URLs. A simple robots.txt fix seems to do the job for this traffic, although one bot with "User-agent: Feedfetcher" is still hitting the site. Obvious in hindsight but maybe it will help someone else.

Upvotes: 2

Related Questions