CODEWITHSUNDEEP
pythonredisflaskredis-py
Ptrkcon
Ptrkcon

Reputation: 1505

Redis page counter

I am writing a flask app that will count page views from multiple websites. I've decided to use Redis and Redispy but I am having a hard time deciding how to structure. Originally I tried to have a something like this 

redis.set("date:YYYYMMDD:site:sitename", 1)

I want to be able to query by date, or by site name and display the count value. I tried to use .keys to query by date or by site name but the REDIS docs say to avoid using keys.
So then I thought maybe I can use redis hashes:

redis.hset("date:YYYYMMDD", "site", "sitename")
redis.hset("counter", 1)

Ultimately I want to be able run reports on site counters by site name, by date, or display all values from all dates. I can't seem to find the right structure to allow me get what I want.

Any suggestions would be greatly appreciated! I have not used REDIS before.

Upvotes: 1

Views: 3662

Answers (2)

The Real Bill
The Real Bill

Reputation: 15793

For overall incrementing counters such as described, a structure along the lines of this answer would be a proven way of doing it. The answer given has more detail son the use of sorted sets for time series data such as this. It even has methods for rolling data up (maybe you also want to report on monthly data).

If, as your examples appear to indicate, your resolution is daily then a structure which stores data in a sorted set with the site/domain name in the key and uses a daily timestamp and [ZINCRBY][2] would serve you well.

With regards to keys ... don't. Simply don't.

As an alternative, you sets to store the keys when using domains/sitenames as your identifier. Whenever you run a ZINCRBY for a site/domain name also do an SADD operation on an "index key" such as sites-with-traffic:YYYY:MM:DD. That way you can pull the set of unique sites for that day and build your queries for each one from there.

In your description you say you may want to report on given days. The way to do this with Redis is to simply query for the key data would be in if it were there. If you get nothing back, the traffic for that period is 0. Don't query when you have a set of parameters - use them. The only one of these query patterns you listed which would be possibly unknown would be the aforementioned site/domain name and adding those to a set solves that need.

Upvotes: 1

quiver
quiver

Reputation: 4470

For counting, you can use incr/decr command

http://redis.io/commands/incr

When you receive access, increment counter like this:

  > incr counter:site2:20131101
  (integer) 1
  > incr counter:site1:20131103
  (integer) 1
  > incr counter:site1:20131103
  (integer) 2
  > incr counter:site2:20131103
  (integer) 1

When you query counters, first get all keys then sum up all counters in your app.

  > keys counter:site2:* # query by site
  1) "counter:site2:20131101"
  2) "counter:site2:20131103"
  > get "counter:site2:20131101"
  "1"
  > get "counter:site2:20131103"
  "1"
  > keys counter:*:20131103 # query by date
  1) "counter:site1:20131103"
  2) "counter:site2:20131103"
  ...

Upvotes: 0

Related Questions