CODEWITHSUNDEEP
node.jsmongodbmongoosepaginationmapreduce
Manuel Di Iorio
Manuel Di Iorio

Reputation: 3771

Paginating a mongoose mapReduce, for a ranking algorithm

I'm using a MongoDB mapReduce to code a ranking feed algorithm, it almost works but the latest thing to implement is the pagination. The map reduce supports the results limitation but how could I implement the offset (skipping) based e.g. on the latest viewed _id of the results, knowing that I'm using mongoose?

This is the procedure I wrote:

o = {};

o.map = function() {
    //log10(likes+comments) / elapsed hours from the post creation
    emit(Math.log(this.likes + this.comments + 1) / Math.LN10 / Math.abs((now - this.createdAt) / 6e7 + 1), this);
};

o.reduce = function(key, values) {
    //sort the values, when they have the same score
    values.sort(function(a, b) {
        a.createdAt - b.createdAt;
    });

    //serialize the values, because mongoose does not support multiple returned values
    return JSON.stringify(values);
};

o.scope = {now: new Date()};
o.limit = 15;

Posts.mapReduce(o, function(err, results) {
    if (err) return console.log(err);
    console.log(results);
});

Also, if the mapReduce it's not the way to go, do you suggest other on how to implement something like this?

Upvotes: 1

Views: 248

Answers (1)

Zlatko
Zlatko

Reputation: 19578

What you need is a page delimiter which is not the id of the latest viewed as you say, but your sorting property. In this case, it seems to be the formula Math.log(this.likes + this.comments + 1) / Math.LN10 / Math.abs((now - this.createdAt) / 6e7 + 1).

So, in your mapReduce query needs to hold a where value of that formula above. Or specifically, 'formula >= . And also it needs to hold the value of createdAt at the last page, since you don't sort by that. (Assuming createdAt is unique). So yourqueryof mapReduce would saywhere: theFormulaExpression, createdAt: { $lt: lastCreatedAt }`

If you do allow multiple identical createdAt values, you have to play a little outside of the database itself.

So you just search by formula.

Ideally, that gives you one element with exactly that value, and the next ones sorted after that. So in reply to the module caller, remove this first element off the array (and make sure you actually ask for more results then you need because of this).

Now, since you allow for multiple similar values, you need another identifying prop, say, object id or created_at. Your consumer (caller of this module) will have to provide both (last value of the score, createdAt of the last object). Say you have a page split exactly in the middle - one or more objects is on the previous page, another set on the next . You'd have to not simply remove the top value (because that same score is already served on the previous page), but possibly several of them from the top.

Then it goes really crazy, because potentially your whole page was already served - compare the _ids, look for the first one after the one your module caller has provided you with. Or look into the data and determine how many matching values like that are there, try to get at least as many more values from mapReduce then you have on your actual page size.

Aside from that, I would do this with aggregation instead, it should be much more preformant.

Upvotes: 2

Related Questions