CODEWITHSUNDEEP
htmlajaxindexing
holian
holian

Reputation: 755

Prevent google from indexing ajax loaded content

On our site we load identical content via Ajax calls (when the users click on the menu, just to prevent reloading the entire page again, so as to improve user experience).

So this is works well, but actually this Ajax loaded content is actually a copy of the original content.

May I prevent Google from indexing this content?

http://dinox-h.hu/en/gallery.php

In the left menu you can see the links:

For example:

http://dinox-h.hu/puffer_tartalyok_galeria.php?ajax=1

Upvotes: 3

Views: 5945

Answers (1)

Ian Clark
Ian Clark

Reputation: 9357

Try adding the following on your Ajax-delivered pages:

<meta name="robots" content="noindex,nofollow" />

This will tell site crawlers to not crawl the page. You could also add the pages in robots.txt, like this:

User-agent: *
Disallow: /*?ajax=1

That would block any URL with ?ajax=1 from being indexed (providing a robot honours your robots.txt). A better solution would also involve creating a sitemap and telling various search engines about it.

Edit A better way of delivering Ajax content IMO would be to send the following header when requesting your pages via Ajax:

X-Requested-With: XMLHttpRequest

jQuery will do this by default, so provided you can check for it on the server side, you could deliver your usual content e.g. without the template. You could then very easily deliver different content from the same URL depending on what the type of request is. This should also solve your crawling issue as I doubt a crawler would stumble across it.

Upvotes: 6

Related Questions