CODEWITHSUNDEEP
mongodbmapreduce
Paté
Paté

Reputation: 1964

MapReduce and grouping by date on MongoDb

I'm trying to test out mongoDb to manage our performance logs.

For this test I inserted 10M rows in a collection using the following format

{ "_id" : ObjectId("4e9d3cc4621dc1dc11000000"), "date" : "Thu Oct 13 2011 15:37:21 GMT+0200 (CEST)", "loadtime" : 0.07, "msg" : "Lorem ipsum message" }

Now I would like to be able to get average load time by day.

So from what I understood I need to do 2 pass of MapReduce.

the first one would be to create a collection of days.

So I tried 

map = function() {
  day = Date.UTC(this.date.getFullYear(), this.date.getMonth(), this.date.getDate());

  emit({day: day}, {count: 1});
}

reduce = function(key, values) {
  var count = 0;

  values.forEach(function(v) {
    count += v['count'];
  });

  return {count: count};
}

Refering to http://cookbook.mongodb.org/patterns/unique_items_map_reduce/

But this kills my mongodb server after a few seconds.

db.loadTime.mapReduce(map, reduce, {out: 'days'});
Tue Oct 18 11:57:28 query failed : test.$cmd { mapreduce: "loadTime", map: function () {
    day = Date.UTC(this.date.getFullYear(), this.date.ge..., reduce: function (key, values) {
    var count = 0;
    values.forEach(functio..., out: "days" } to: 127.0.0.1
Tue Oct 18 11:57:28 Error: error doing query: failed (anon):1509

Here is my error log

Tue Oct 18 11:56:08 [conn1] CMD: drop test.tmp.mr.mapreduce_1318931768_1_inc
        55800/10000000  0%
        112800/10000000 1%
        171400/10000000 1%
        229600/10000000 2%
        288600/10000000 2%
        345600/10000000 3%
        404100/10000000 4%
        462900/10000000 4%
        522000/10000000 5%
        579100/10000000 5%
        629200/10000000 6%
        677000/10000000 6%
        724200/10000000 7%
        767500/10000000 7%
        818600/10000000 8%
        864300/10000000 8%
        921300/10000000 9%
        972200/10000000 9%
        1021600/10000000    10%
        1070700/10000000    10%
        1115600/10000000    11%
        1163600/10000000    11%
        1217400/10000000    12%
        1269100/10000000    12%
        1313300/10000000    13%
        1366200/10000000    13%
Tue Oct 18 11:57:28 Got signal: 11 (Segmentation fault).

Tue Oct 18 11:57:28 Backtrace:
0x843a16d 0x842dbcd 0x741400 0x1eadcd 
 /usr/lib/mongodb/mongod(_ZN5mongo15printStackTraceERSo+0x2d) [0x843a16d]
 /usr/lib/mongodb/mongod(_ZN5mongo10abruptQuitEi+0x3ed) [0x842dbcd]
 [0x741400]
 /usr/lib/xulrunner-2.0/libmozjs.so(+0xdadcd) [0x1eadcd]

Am I going in the right direction ?

Upvotes: 0

Views: 4225

Answers (1)

Parvin Gasimzade
Parvin Gasimzade

Reputation: 26032

I also got same problem and solve it as follows.

Instead of 

values.forEach(function(v) {
   count += v['count'];
});

use 

for(v in values){
   count += v['count'];
}

Upvotes: 4

Related Questions