MongoDB MapReduce--is there an Aggregation alternative?

Question

I've got a collection with documents using a schema something like this (some members redacted):

{
    "_id" : ObjectId("539f41a95d1887b57ab78bea"),
    "answers" : {
        "ratings" : {
            "positivity" : [ 
                2, 
                3, 
                5
            ],
            "activity" : [ 
                4, 
                4, 
                3
            ],
    },
    "media" : [ 
        ObjectId("537ea185df872bb71e4df270"), 
        ObjectId("537ea185df872bb71e4df275"), 
        ObjectId("537ea185df872bb71e4df272")
    ]
}

In this schema, the first, second, and third positivity ratings correspond to the first, second, and third entries in the media array, respectively. The same is true for the activity ratings. I need to calculate statistics for the positivity and activity ratings with respect to their associated media objects across all documents in the collection. Right now, I'm doing this with MapReduce. I'd like to, however, accomplish this with the Aggregation Pipeline.

Ideally, I'd like to $unwind the media, answers.ratings.positivity, and answers.ratings.activity arrays simultaneously so that I end up with, for example, the following three documents based on the previous example:

[
    {
        "_id" : ObjectId("539f41a95d1887b57ab78bea"),
        "answers" : {
            "ratings" : {
                "positivity" : 2,
                "activity" : 4
            }
        },
        "media" : ObjectId("537ea185df872bb71e4df270")
    },
    {
        "_id" : ObjectId("539f41a95d1887b57ab78bea"),
        "answers" : {
            "ratings" : {
                "positivity" : 3
                "activity" : 4
            }
        },
        "media" : ObjectId("537ea185df872bb71e4df275")
    },
    {
        "_id" : ObjectId("539f41a95d1887b57ab78bea"),
        "answers" : {
            "ratings" : {
                "positivity" : 5
                "activity" : 3
            }
        },
        "media" : ObjectId("537ea185df872bb71e4df272")
    }
]

Is there some way to accomplish this?

Answer 1

The current aggregation framework does not allow you to do this. Being able to unwind multiple arrays that are know to be the same size and creating a document for the ith value of each would be a good feature.

If you want to use the aggregation framework you will need to change your schema a little. For example take the following document schema:

{
    "_id" : ObjectId("539f41a95d1887b57ab78bea"),
    "answers" : {
        "ratings" : {
            "positivity" : [ 
                {k:1, v:2}, 
                {k:2, v:3}, 
                {k:3, v:5}
            ],
            "activity" : [ 
                {k:1, v:4}, 
                {k:2, v:4}, 
                {k:3, v:3}
            ],
    }},
    "media" : [ 
        {k:1, v:ObjectId("537ea185df872bb71e4df270")}, 
        {k:2, v:ObjectId("537ea185df872bb71e4df275")}, 
        {k:3, v:ObjectId("537ea185df872bb71e4df272")}
    ]
}

By doing this you are essentially adding the index to the object inside the array. After this it's just a matter of unwinding all the arrays and matching on the key.

db.test.aggregate([{$unwind:"$media"},
{$unwind:"$answers.ratings.positivity"},
{$unwind:"$answers.ratings.activity"},
{$project:{"media":1, "answers.ratings.positivity":1,"answers.ratings.activity":1,
    include:{$and:[
                  {$eq:["$media.k", "$answers.ratings.positivity.k"]},
                  {$eq:["$media.k", "$answers.ratings.activity.k"]}
            ]}}
},
{$match:{include:true}}])

And the output is:

[ 
        {
            "_id" : ObjectId("539f41a95d1887b57ab78bea"),
            "answers" : {
                "ratings" : {
                    "positivity" : {
                        "k" : 1,
                        "v" : 2
                    },
                    "activity" : {
                        "k" : 1,
                        "v" : 4
                    }
                }
            },
            "media" : {
                "k" : 1,
                "v" : ObjectId("537ea185df872bb71e4df270")
            },
            "include" : true
        }, 
        {
            "_id" : ObjectId("539f41a95d1887b57ab78bea"),
            "answers" : {
                "ratings" : {
                    "positivity" : {
                        "k" : 2,
                        "v" : 3
                    },
                    "activity" : {
                        "k" : 2,
                        "v" : 4
                    }
                }
            },
            "media" : {
                "k" : 2,
                "v" : ObjectId("537ea185df872bb71e4df275")
            },
            "include" : true
        }, 
        {
            "_id" : ObjectId("539f41a95d1887b57ab78bea"),
            "answers" : {
                "ratings" : {
                    "positivity" : {
                        "k" : 3,
                        "v" : 5
                    },
                    "activity" : {
                        "k" : 3,
                        "v" : 3
                    }
                }
            },
            "media" : {
                "k" : 3,
                "v" : ObjectId("537ea185df872bb71e4df272")
            },
            "include" : true
        }
    ]

Doing this creates a lot of extra document overhead and may be slower than your current MapReduce implementation. You would need to run tests to check this. The computations required for this will grow in a cubic way based on the size of those three arrays. This should also be kept in mind.