CODEWITHSUNDEEP
phpbrowserseobots
Geshan
Geshan

Reputation: 1141

Access a website by a bot but only after setting a variable by a browser/human

We are working on a website of a brewery using Drupal. As minors under 18 are not allowed to view the website, any page accessed is passed through a module (legalage) and if the user clicks "continue" so as to confirm his/her age being more than 18 the site is shown to the user saving the legal age as true.

The problem is when search engine bots access the pages, they are also redirected to the legal age verification page. I tried the following script to determine bot or browser in PHP:

$agent = $_SERVER['HTTP_USER_AGENT'];
if(( eregi("bot", $agent) || eregi("slurp", $agent) )) {

$_SESSION['legalage'] = true;
drupal_goto($_REQUEST['destination']);

}

but its does not seem to work. If anyone can suggest me on how to handle this problem so that the browsers/humans have to pass through the age requirement continue thing and bots can directly access the content. Thank you in advance.

Upvotes: 1

Views: 447

Answers (4)

Salman Arshad
Salman Arshad

Reputation: 272106

Your logic is correct. I am however not sure if bots accept and echo session cookies. I suggest that on the pages deep inside that require $_SESSION['legalage'] = true; you also add code to ignore this logic for bots.

FYI, Google Webmaster Tools just added a handy new feature that shows you the actual content sent by the server when Google bot accesses it. Go ahead and use it!

Upvotes: 1

Sabeen Malik
Sabeen Malik

Reputation: 10880

one possibility might be to put a javascript redirect instead of a header redirect .. bots wont be able to process it and normal ppl will be redirected .. however there will always be a group of ppl who will hava javascript disabled ... but that group would be small and generally over 18 :)

Upvotes: 1

Ben James
Ben James

Reputation: 125167

Storing session data usually requires cookies on the client. If the bot doesn't accept cookies, it won't work. (Unless you enable use_trans_sid, which will add a session ID querystring to every URL.)

Try doing the user agent check around the code that performs the redirect back to the age verification page, rather than on the age verification page itself.

As an aside, don't use eregi -- it's deprecated. Use the perl-compatible regular expression functions instead.

Upvotes: 4

Lizard
Lizard

Reputation: 44992

Detecting bots/spider/crawlers isn't an exact science I have a PHP array that I have used in the past and checked that against the user agent. The array contains most if not all the main spiders / crawlers you would want to let visit you site. Would this be of interest to you? Or are you looking for somthing else?

Upvotes: 0

Related Questions