CODEWITHSUNDEEP
asp.netweb-applicationsstatistics
v s
v s

Reputation: 559

asp.net avoiding online user count increment when request is made by google bots or other crawlers

We are doing 

    Appliation["OnlineUsers"] += 1 in Session Start,

However whenever google bot scans our site this number increases very fast and we want to avoid this in general. We want to increase this number only for real user.

Let us know any other suggestions

Upvotes: 2

Views: 282

Answers (2)

Andrew Morton
Andrew Morton

Reputation: 25013

If you want a regular expression you can use to match many spiders, and one that matches (what I've found to be) common attack attempts:

    '' portions of the browser string which indicate a spider
    Dim re As New Regex("bot|spider|slurp|crawler|teoma|DMOZ|;1813|findlinks|tellbaby|ia_archiver|nutch|voyager|wwwster|3dir|scooter|appie|exactseek|feedfetcher|freedir|holmes|panscient|yandex|alef|cfnetwork|kalooga|Charlotte|isara|butterfly|kilomonkey|larbin|postrank|webcollage|netcraft|Netintelligence|baypup|dragonfly|EmailSiphon|ExactSearch|LucidMedia|Mail\.Ru|MSIndianWebcrawl|PycURL|Python-urllib|Qryos|Robozilla|SBIder|StackRambler|BoardReader|scoutjet|allrati|192\.comAgent|cizilla|dnsdigger|qwant", RegexOptions.Compiled Or RegexOptions.IgnoreCase)

    '' attacks
    Dim badRe As New Regex("morfeus|nv32ts|dragostea|zmeu|DataCha0s|GT::WWW|CZ32ts", RegexOptions.IgnoreCase Or RegexOptions.Compiled)

then

If re.IsMatch(userAgentString) Then ' it's a spider

and similarly for attack attempts.

Upvotes: 2

Ryan McDonough
Ryan McDonough

Reputation: 10012

I would suggest building in a check for the USER AGENT in your code, all bots have to identify themselves as such so you could do a check in this sense.

However you would have to do the count some other way rather than checking Session start.

Upvotes: 1

Related Questions