CODEWITHSUNDEEP
robots.txt
Kirzilla
Kirzilla

Reputation: 16596

Disallow all for all user agents except one user agent?

How to disallow all for all user agents except one user agent? For example disallow all for all user agent, but allow for Googlebot only?

Upvotes: 0

Views: 9410

Answers (2)

user24721076
user24721076

Reputation: 1

SUDO DISALLOW ALL HOSTS AND USER AGENTS ESPECIALLY BOTS. Only local owners shal be allowed in local drive and directory. To preserve human life and psychological safety.

In that this organization is not currently licensed due to the reckless disregard for human safety, it is evident to me that they shall not make ANY decisions on my behalf or that of my love ones.

Upvotes: 0

Kent Kostelac
Kent Kostelac

Reputation: 2466

User-agent: *
Disallow: /

User-agent: google
Allow: /

This sample robots.txt tells crawlers that if they are not with google. then it is preferred they don't crawl your site. While google has been given the greenpass to crawl anything on the site.

This file should be stored at www.example.com/robots.txt.

Please read up on robots.txt

Upvotes: 1

Related Questions