CODEWITHSUNDEEP
djangoseorobots.txt
Enirsa
Enirsa

Reputation: 99

Trailing slash in the URL for robots.txt

I'm developing a website using Django, and Django really likes its trailing slashes so it's easier to give in than to go against it. I planning to also serve the robots.txt file via Django, so I'm wondering:

Would it matter for search bots if robots.txt file was located at example.com/robots.txt/ and example.com/robots.txt would redirect to the URL with the trailing slash? Or does this make no difference at all?

UPDATE: According to Robots.txt Specifications, at least five redirect hops are allowed, so I'm in the clear.

Upvotes: 0

Views: 1090

Answers (1)

Wiktoria Tranowska
Wiktoria Tranowska

Reputation: 46

According to https://developers.google.com/search/reference/robots_txt The URL for the robots.txt file is - like other URLs - case-sensitive so you should use robots.txt only without trailing slash (and redirection).

Upvotes: 1

Related Questions