CODEWITHSUNDEEP
httpsecuritynginxcloudflaregooglebot
Innmedia
Innmedia

Reputation: 13

How can I set Cloudflare firewall to block fake Google bots

I know that these ip's are fake Google bots and I'd like to block them. Ip isn't static, so I'd like to block range in Cloudflare.

However it doesn't work. Here is the sample log of Google fake bot: https://gyazo.com/f4bd7182923c6caa92cf95a7f84116cd

Here are fake bot IPs:

162.158.74.61
162.158.74.175
162.158.75.188
162.158.75.68
162.158.75.230
162.158.75.242
162.158.75.26
162.158.74.205
172.68.58.213
172.68.58.141
172.68.58.21
172.68.59.40
172.68.59.184
108.162.216.176
108.162.216.248
108.162.216.92

Here is my current Cloudflare configuration: https://gyazo.com/91af7d0964621c95d5cc5a462724cb9a

Any ideas of how it should be done?

Upvotes: 1

Views: 1153

Answers (2)

krówek
krówek

Reputation: 44

Add to /etc/nginx/nginx.conf in http block:

geo $fakebotip {
    default 0;
    108.162.0.0/16 1;
    141.101.0.0/16 1;
    162.158.0.0/16 1;
    172.68.0.0/16 1;
    172.69.0.0/16 1;
}

And then to /etc/nginx/sites-available/your-site.pl:

if ($fakebotip = 1) {
    set $naughtyBot A;
}
if ($http_user_agent ~ (Googlebot|SemrushBot|AhrefsBot|bingbot|Qwantify|YandexBot|YandexImages|DotBot|linkfluence|Mediatoolkitbot|BoardReader) ) {
    set $naughtyBot "${naughtyBot}B";
}
if ($naughtyBot = AB) {
    return 404;
}

Check nginx config: sudo nginx -t

If it's all good, reload nginx: sudo systemctl reload nginx

Upvotes: 1

Rjj
Rjj

Reputation: 141

You can block all bots via robots.txt and by putting this

User-agent: * 
Disallow: /

It will block all the bots crawling in your website

Also if you want to let a bot, example googlebot put this

User-agent: Googlebot
Allow: /

Upvotes: 0

Related Questions