CODEWITHSUNDEEP
regex.htaccessmod-rewriteuser-agent
mseifert
mseifert

Reputation: 5670

.htaccess not blocking specified user agent

UPDATE I found the Firefox addon: User Agent Switcher which allows me to test with a different user agent. Surprise, it blocked the user agent as it should. I wonder why the 12 or so requests which took place after my first block attempt via .htaccess somehow slipped in. Were the page requests already in process? Oh well, at least they are working.

In my .htaccess I have a php script that checks for offending user agent strings and dynamically writes to .htaccess a RewriteCond to immediately block the user agent.

I tried to block the agent Mozilla/5.0 (Auto Shell Spider) by writing the following condition RewriteCond %{HTTP_USER_AGENT} ^.*(Mozilla/5\.0\ \(Auto\ Shell\ Spider\)).*$ [NC] but my log showed the user agent continued to access pages (another 10+ over the next 2 seconds). I've tested the rule with preg_match() and it matches and therefore should block the user agent. What is happening? I am under the impression that I don't need an AND in the RewriteCond since each has its own RewriteRule. I don't believe there is caching of .htaccess - changes should take effect immediately. Is php regex different from htaccess?

Any help would be appreciated.

Here is a more complete set of .htaccess instructions:

RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)$ index.php

# Block spiders: 
    RewriteCond %{HTTP_USER_AGENT} ^.*(Baiduspider|360Spider|Sogou|Sosospider|Yandex|NaverBot|Yeti|moget|ichiro|RedBot|AhrefsBot|xovibot).*$ [NC]
    RewriteRule .* - [R=403,L]
Deny from 59.61.184.100
Deny from 59.152.240.71

# Block User Agent -  NOT WORKING?
RewriteCond %{HTTP_USER_AGENT} ^.*(Mozilla/5\.0\ \(Auto\ Shell\ Spider\)).*$ [NC]
RewriteRule .* - [R=403,L]

When using php preg_match, 

$text =  "Mozilla/5.0 (Auto Shell Spider)";
preg_match('`^.*(Mozilla/5\.0\ \(Auto\ Shell\ Spider\)).*$`', $text, $acell);    
echo "<pre>";var_dump($acell);echo "</pre>";

I get a match as follows:

array (size=2)
  0 => string 'Mozilla/5.0 (Auto Shell Spider)' (length=31)
  1 => string 'Mozilla/5.0 (Auto Shell Spider)' (length=31)

Upvotes: 0

Views: 2030

Answers (1)

anubhava
anubhava

Reputation: 785156

Try reordering your rules like this:

Deny from 59.61.184.100
Deny from 59.152.240.71

RewriteEngine On

# Block spiders: 
RewriteCond %{HTTP_USER_AGENT} ^.*(Baiduspider|360Spider|Sogou|Sosospider|Yandex|NaverBot|Yeti|moget|ichiro|RedBot|AhrefsBot|xovibot).*$ [NC]
RewriteRule ^ - [R=403,L]

# Block User Agent -  NOT WORKING?
RewriteCond %{HTTP_USER_AGENT} "Mozilla/5\.0 \(Auto Shell Spider\)" [NC]
RewriteRule ^ - [R=403,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^ index.php [L]

Upvotes: 1

Related Questions