user2801966
user2801966

Reputation: 468

How do i tell google to not crawl a domain completly

I have a site in opencart say abc.com and i have opened a multi store with it xyz.com and i have found that google has started crawling xyz.com too which i don't want . Both the domains are pointing to the same directory so i suppose there can only be one robots.txt file . I am not very experienced with google webmaster or SEO so please tell me how can i tell google to ignore all the urls starting with xyz.com .

Upvotes: 1

Views: 253

Answers (1)

You Old Fool
You Old Fool

Reputation: 22941

If you are using Apache and mod_rewrite you can add a rewrite rule to serve a different robots.txt file for xyz.com:

RewriteCond %{HTTP_HOST} xyz.com$ [NC]
RewriteRule ^robots.txt robots_xyz.txt [L]

Then create robots_xyz.txt:

User-agent: *
Disallow: /

Upvotes: 1

Related Questions