Reputation: 1244
My client wants me to make his website completely invisible in search results, especially in Google search engine. The website will be a simple presentation one-page site, which should be normally accessible (no locks, no passwords).
I have several ideas which might certainly work and one where I am not sure.
What might certainly work:
Adjust robots.txt with following string:
User-agent: *
Disallow: /
Add nofollow meta-tag in page head:
<meta name="robots" content="noindex,nofollow" />
I can detect user-agent strings in all requests, make list of known robots and scrapers and if there will be match I can kick them out. This should work too I guess.
This link should be helpful in case of bots detection, these guys seems to provide API for known user agent strings.
Domain choice can influence search results too:
My client prefer to stay unlisted in certain countries. I noticed that Google prefers to serve us results according to our IP adress, so if your IP is located in Sweden than you can be sure that most results are with *.se domain. Commercial domains like *com, *.net, *.org seems to be visible in all cases, it make sense. So it brought me to question, how domain choice can influence search results. For example if I prefer to stay hidden for German people I should not take .de domain of course and I should avoid commercial domains. Ok I can choose another country, but it seems to be weird, I would prefer to take new domains extension like (.club, *.art, *.shop, *.name etc.) but here I am not sure how will Google treat those domains. If they behave like commercial domain then I should stay with other country-code domain.
I hope that my intentions are obvious here. I would be glad for your valuable advices here, if there is something more what I can do, or if anybody can bring more light in domain problem I would be very pleased.
Upvotes: 1
Views: 1645
Reputation: 316
The options you have listed will work well, especially the robots.txt. But another to consider would be using the google webmaster tools to add a no follow to the page / website. Works well as it allows your client to add no follows to other pages down the line also.
Upvotes: 2