CODEWITHSUNDEEP
dictionarymapreducecouchbase
Meta_data
Meta_data

Reputation: 608

Couchbase Reduce function

I am trying to learn how to use map reduce functions with Couchbase. until now i created reports engines based on SQL using Where with multi terms (adding and subtracting terms) and to modify the group part.

I am trying to create this report engine using views.

my problem is how to create a report that enable users to dive in and find more and more data, getting all the way to individual ip stats. For example. how many clicks where today ? which traffic source ? what did they see? which country ? and etc..

My basic doc for this example look like this:

"1"
{
 "date": "2014-01-13 10:00:00",
  "ip": "111.222.333.444",
  "country": "US",
  "source":"1",
}
"2"
{
 "date": "2014-01-13 10:00:00",
  "ip": "555.222.333.444",
  "country": "US",
  "source":"1",
}
"3"
{
 "date": "2014-01-13 11:00:00",
  "ip": "111.888.888.888",
  "country": "US",
  "source":"2",
}
"4"
{
 "date": "2014-01-13 11:00:00",
  "ip": "111.777.777.777",
  "country": "US",
  "source":"1",
}

So i want to allow the user to see at the first screen , how many clicks per day there are at this site. so i need to count the amount of clicks. simple map/reduce: MAP:

function (doc, meta) {
     emit(dateToArray(doc.date),1);
}
Reduce:
_count

group level 4, group true

will create the sum of clicks per hour.

Now if i want to allow a break down of countries, so i need a dynamic param to change.. from what i am understand it can only by the group level.. so assume i have added this to the emit like this:

emit([dateToArray(doc.date),source],1);

and then grouping level 5 will allow this divide, and using the key too focus on a certein date.. but what if i need to add a county break down? adding this to the emit again? this seem to be a mess, also if i will want to do a country stats before the source.. is there any smarter way to do this?

Second part...

What if i want to get the first count as follow:

[2014,1,28,10] {ip:"555.222.333.444","111.222.333.444","count":"2"}

i want to see all the ips that are counted for this time... how should i write my reduce function?

this is my current state that doesnt work..

function(key, values, rereduce) {
  var result = {id: 0, count: 0};
  for(i=0; i < values.length; i++) {
    if(rereduce) {
        result.id = result.id + (values[i]).ip +',';
        result.count = result.count + values[i].count;
    } else {
        result.id = values.ip;
        result.count = values.length;
    }
  }
  return result;

i didnt get the answer format i was looking for..

i hope this is not to messy and that you could help me with this..

thanks!!

Upvotes: 2

Views: 2306

Answers (1)

theMayer
theMayer

Reputation: 16177

For the first part of your question, I think you are on the right track. That is how you break down views to enable coarse drill down. However, it is important to remember that views are not intended to store your entire documents, nor are they necessarily going to be able to give you a clean cut swatch of data. You probably will need to do fine-filtering within the access layer of your code (using Linq perhaps).

For the second part of your question, a reduce is not the appropriate mechanism to accomplish this. Reduce values have a very finite (and limited) size and will crash the map/reduce engine once they get too big. I suspect you have experimented with that and discovered this for yourself.

The way you worded the question, it seems like you wish to search for all IP addresses that have been counted "X" number of times. This cannot be accomplished directly in Couchbase's map/reduce architecture; however, if you simply want the count for a given IP address, that is something the map/reduce framework has built-in (just use Date + IP as a key).

Upvotes: 1

Related Questions