CODEWITHSUNDEEP
html
zeckdude
zeckdude

Reputation: 16163

How can I ensure search engines don't index my site during development?

I am in the process of developing a website. I am using much example placeholder text as well as dummy images. I want to ensure that Google does not index my pages while I am still in development.

Is there any code I can add to my pages to ensure search engines do not index my site at this time?

Upvotes: 2

Views: 4063

Answers (8)

citynorman
citynorman

Reputation: 5292

Add meta-tag <meta name="robots" content="noindex"> see Google support

Upvotes: -1

ObscureRobot
ObscureRobot

Reputation: 7336

Try using robots.txt. But beware that it is not a security measure. Some bots will completely ignore it.

Some history on robots.txt.

As others have said, you really shouldn't be developing your site in public. A better solution is to develop on a local machine that isn't internet-facing, or restrict access to the portions of your site that are under development.

Upvotes: 1

FloydThreepwood
FloydThreepwood

Reputation: 1585

If you have to develop online... ;) There are two easy ways to do this.

Create a robots.txt file (http://www.robotstxt.org/

Use htaccess for password access only (http://en.wikipedia.org/wiki/.htaccess)

Upvotes: 2

Russell Dias
Russell Dias

Reputation: 73292

Ideally, you shouldn't be developing a site on the public front in the first place!

But you can edit your robots.txt like so:

User-agent: *
Disallow: /

Upvotes: 5

Aurelio De Rosa
Aurelio De Rosa

Reputation: 22152

You can find some suggest to your problem in a similar question that you can find here:

https://webmasters.stackexchange.com/questions/20964/best-way-to-take-down-site-for-a-long-period-while-i-develop-it/20965

Basically you can use robots.txt, create a private folder or Return 403 (use HTTP authentication) for all URLs on your development site.

Upvotes: 0

redache
redache

Reputation: 21

adding a robots.txt file should stop sites from indexing it.

You can find more information here http://www.robotstxt.org/ but basically it tells the web robots/spiders to ignore the site.

Hope this helps!

Upvotes: 0

animuson
animuson

Reputation: 54729

Create a file called "robots.txt" at the root directory of your website and inside it put:

Disallow: /

And upload it. Google will follow your robots.txt file, but not all crawlers will pay attention to it.

Upvotes: 0

Richard Andrew Lee
Richard Andrew Lee

Reputation: 1177

set the robots.txt

http://www.robotstxt.org/

this should impact all the search engines you care about.

(or you could temporarily password protect the site.)

Upvotes: 1

Related Questions