Léa Massiot
Léa Massiot

Reputation: 2018

Where to put sitemap.xml and robots.txt in case of a Tomcat server with several Webapps

I have a Tomcat server and several Webapps (webapp1, webapp2, ...).

Where do the sitemap.xml and robots.txt files should be put?

Do I need one sitemap.xml file and one robots.txt file for each Webapp?

Shall I put sitemap.xml and robots.txt under respectively:

"where_tomcat_is_installed/webapps/webapp1/",

"where_tomcat_is_installed/webapps/webapp2/",

...

Shall I disallow /WEB-INF/lib/?

If not, is it sufficient to put the following contents into robots.txt:

User-agent: *

Upvotes: 3

Views: 2608

Answers (1)

Olaf Kock
Olaf Kock

Reputation: 48077

robots.txt needs to be in the root folder, not nested in some webapp (or multiple of these). It references sitemap.xml, so that file has to be whereever robots.txt claims it to be. No robot will ever check for /some/path/to/robots.txt, they'll only go for robots.txt

If you have multiple webapps on one server, you'll have to combine their individual configurations in a single robots/sitemap (well, single for each) file.

Upvotes: 4

Related Questions