CODEWITHSUNDEEP
robots.txt
John Doucette
John Doucette

Reputation: 4540

Can I use a `robots.txt` file for a subdirectory on my school's domain?

I own some webspace which is registered with a University. Google has unfortunately found my CV (resume) on the site, but has mis-indexed it as a scholarly publication, which is screwing up things like citation counts on Google Scholar. I tried to upload a robots.txt into my local subdirectory. The problem is that google ignores this file, and instead uses the rules listed for the school domain.

That is, the url looks like

www.someschool.edu/~myusername/mycv.pdf

I have uploaded a robots.txt, which can be found here

www.someschool.edu/~myusername/robots.txt

And Google is ignoring it and instead using the robots.txt for the school's domain

www.someschool.edu/robots.txt

How can I make Googlebot ignore my CV?

Upvotes: 2

Views: 934

Answers (1)

cha0site
cha0site

Reputation: 10717

Sadly, robots.txt is defined to be whatever you get when you GET /robots.txt, so you can't use it for your subdirectory.

What you can do is use the X-Robots-Tag HTTP header, if you can use custom .htaccess files. Here's Google's documentation on X-Robots-Tag.

Upvotes: 3

Related Questions