Reputation: 750
I have discovered some of my dev sites have been indexed by Google. Somehow the robots.txt was removed. How can I tweak my .htaccess to give a 410 status if the visitor is the google spider bot OR the visit is a referral from google?
Upvotes: 0
Views: 167
Reputation: 18671
You can use in your .htaccess
:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_REFERER} google
RewriteRule ^ - [L,R=410]
Upvotes: 1