CODEWITHSUNDEEP
seogooglebotweb-crawler
Panagiotis Panagi
Panagiotis Panagi

Reputation: 10077

How to tell Google bot that certain links no longer exist

In the first days of a website, I made a mistake in the generation of some links; following them outputs a database error.

Google bot has attempted to follow those links and now they appear as crawl errors in webmasters tools. Although I have since corrected the error, google still tries to crawl them.

How to tell Google that such links no longer exist and that they lead to no where? I can't do a 301 redirect cause that would say that the links are valid and its ok to index them.

Upvotes: 2

Views: 1895

Answers (2)

Robb
Robb

Reputation: 3851

If the page is gone with no logical replacement then you should normally return a 404 error, in this case however I would consider returning 410 Gone - which indicates that the page has been permanently removed never to return and should be purged from googles indices.

Its probably worth greping your existing codebase for the url's in question to ensure that no links remain to them.

As for the 301's 301 doesn't mean the link is valid, 301 is moved permanently. This tells google that the page it used to look for here now lives elsewhere and directs google to the new spot. Not applicable in your case from the sounds of things but subtly different from the way you described it.

For more information see http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Upvotes: 5

Kit Scuzz
Kit Scuzz

Reputation: 311

I would just mark them in your robots.txt

http://www.robotstxt.org/robotstxt.html

Upvotes: 1

Related Questions