CODEWITHSUNDEEP
apachesecuritybots
Maroof Mandal
Maroof Mandal

Reputation: 531

How to block Bots excluding crawlers from accessing my site?

A bot crawls my site every few minutes or hours and my site becomes unavailable due to resource shortage. The bot origins from Russia.

How do I block this bot from accessing my site?

Upvotes: 1

Views: 1796

Answers (2)

Maroof Mandal
Maroof Mandal

Reputation: 531

I have solved he issue.

The solution:

Check Latest visitors to your domain Check the user agent which is visiting consistently.

In my case I found YandexBot

Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)

Now that you have found the bot which is slowing down your server, go ahead and block it. You can either do it with robots.txt or .htaccess

.htaccess method:

Add the following code in it.

#block bad bots with a 403
BrowserMatchNoCase "Yandex" bots
Order Allow,Deny
Allow from ALL
Deny from env=bots

Or

robots.txt method:

Add the following code in it.

User-agent: Yandex
Disallow: /

That's it. You've blocked the bot.

Upvotes: 1

Amit Verma
Amit Verma

Reputation: 41249

Deny directive should be inside the files section, try :

<Files 403.shtml>
order allow,deny
allow from all
deny from xxx.xxx.xx.xxx
</Files>

Alternatively, you could use a mod-rewrite based ip blocking

RewriteEngine on

RewriteCond %{REMOTE_ADDR} ^1\.2\.3\.4\.5$
RewriteRule  - [F]

Upvotes: 0

Related Questions