CODEWITHSUNDEEP
robots.txt
Derfder
Derfder

Reputation: 3324

Should I use #End Robots# or not in robots.txt?

Should I use #End Robots# or not in robots.txt?

I mean does it help to prevent me from getting whitespaces or not?

right now it looks something like that:

User-agent: *
Disallow: /admin/
Disallow: /account/
Disallow: /access-denied/
#End Robots#

Upvotes: 0

Views: 63

Answers (1)

Jon Malcolm
Jon Malcolm

Reputation: 461

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449 seems to suggest that it isn't needed.

Personally I've never added that to any robots file I've deployed and I've never encountered any problems.

Upvotes: 2

Related Questions