CODEWITHSUNDEEP
asp.netweb-applicationswebanalytics
Brandon Montgomery
Brandon Montgomery

Reputation: 6986

Filter out search engine bot traffic

I'm doing custom-rolled view tracking on my website, and I just realize that I totally forgot about search bots hitting the pages. How do I filter out that traffic from my view tracking?

Upvotes: 2

Views: 665

Answers (2)

Oli
Oli

Reputation: 239810

Look at the user-agents. It might seem logical to blacklist, that is filter out all the strings that contain "Googlebot" or other known search engine bots, but there are so many of them, it could well be easiest to just to whitelist: log visitors using a known browser.

Another approach would be to use some JavaScript to do the actual logging (like Google Analytics does). Bots won't load the JS and so won't count toward your statistics. You can also do a lot more detailed logging this way because you can see exactly (down to the pixel - if you want) which links were clicked.

Upvotes: 2

garph0
garph0

Reputation: 1708

You can check the user agent: here there is a nice list. Or you could cross-check with the hits on robots.txt, since all the spiders should read that first and users usually don't.

Upvotes: 0

Related Questions