CODEWITHSUNDEEP
.htaccessbots
nuriye
nuriye

Reputation: 189

Can't block bots in htaccess

I tried to block bad bots via htaccess with this code: I know these are 2 ways to do so, but none of them is working, I still see the bots in the access-log: What am I doing wrong?

RewriteCond %{HTTP_USER_AGENT} ^BLEXBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^SemrushBot [NC,OR]

SetEnvIfNoCase User-Agent "BLEXBot" rotbot
SetEnvIfNoCase User-Agent "SemrushBot" rotbot

<Limit POST GET HEAD PUT>
        Order Allow,Deny
        Allow from all
        Deny from env=rotbot
</Limit>

The entries in the access log look like that:

domain.org:443 46.229.168.142 - - [22/Jul/2019:08:56:26 +0200] "GET /path/to/page/ HTTP/1.1" 403 3801 "-" "Mozilla/5.0 (compatible; SemrushBot/3~bl; +http://www.semrush.com/bot.html)"
domain.org:443 94.130.219.232 - - [22/Jul/2019:08:56:24 +0200] "GET /path/to/page/ HTTP/1.1" 403 760 "-" "Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)"

Upvotes: 0

Views: 1797

Answers (1)

user10946716
user10946716

Reputation:

Fix these rules to:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} ^BLEXBot [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^SemrushBot [NC]
    RewriteRule ^.* - [F,L]
</IfModule>

Upvotes: 2

Related Questions