CODEWITHSUNDEEP
asp.netsearch-engineweb-crawler
Dofs
Dofs

Reputation: 19217

Robots.txt with one site but two domains

I have a website which has two domains added. Both domains point to the root of the website. Is it possible to alter the robots.txt so that one of the domains doesn't get crawled, while the other still does?

Upvotes: 2

Views: 213

Answers (1)

Fenton
Fenton

Reputation: 250952

You ought to perform a 301 redirect to your preferred domain, which tells robots what URL you want crawled.

For example

www.domain1.com

Should 301 redirect to

www.domain2.com

Similarly

www.domain1.com/Contact-Us/

Should 301 redirect to

www.domain2.com/Contact-Us/

Upvotes: 1

Related Questions