Reputation: 2018
I have a Tomcat server and several Webapps (webapp1, webapp2, ...).
Where do the sitemap.xml
and robots.txt
files should be put?
Do I need one sitemap.xml
file and one robots.txt
file for each Webapp?
Shall I put sitemap.xml
and robots.txt
under respectively:
"where_tomcat_is_installed/webapps/webapp1/"
,
"where_tomcat_is_installed/webapps/webapp2/"
,
...
Shall I disallow /WEB-INF/lib/
?
If not, is it sufficient to put the following contents into robots.txt
:
User-agent: *
Upvotes: 3
Views: 2608
Reputation: 48077
robots.txt
needs to be in the root folder, not nested in some webapp (or multiple of these). It references sitemap.xml
, so that file has to be whereever robots.txt
claims it to be. No robot will ever check for /some/path/to/robots.txt
, they'll only go for robots.txt
If you have multiple webapps on one server, you'll have to combine their individual configurations in a single robots/sitemap (well, single for each) file.
Upvotes: 4