Shuffle sub documents in mongoose query

Question

I have following models:

Question Model

var OptionSchema = new Schema({
    correct : {type:Boolean, default:false}
    value : String
});

var QuestionSchema = new Schema({
    value  : String
    , choices : [OptionSchema]
    , quiz : {type:ObjectId, ref:'quizzes'}
    , createdOn : {type:Date, default:Date.now}
    ...
});

var Question = mongoose.model('questions', QuestionSchema);

Quiz Model

var QuizSchema = new Schema({
name : String
, questions : [{type:ObjectId, ref:'questions'}]
,company : {type:ObjectId, ref:'companies'}
...
});

var Quiz = mongoose.model('quizzes', QuizSchema);

Company Model

var CompanySchema = new Schema({
name :String
...
});

I want to shuffle choices of each question per each query, and I am doing It as follows :

shuffle = function(v){
    //+ Jonas Raoni Soares Silva
    //@ http://jsfromhell.com/array/shuffle [rev. #1]

    for(var j, x, i = v.length; i; j = parseInt(Math.random() * i), x = v[--i], v[i] = v[j], v[j] = x);
    return v;
};

app.get('/api/companies/:companyId/quizzes', function(req, res){

    var Query = Quiz.find({company:req.params.companyId});
    Query.populate('questions');
    Query.exec(function(err, docs){
        docs.forEach(function(doc) {
        doc.questions.forEach(function(question) {
            question.choices = shuffle(question.choices);
            })
        });
        res.json(docs);
    });

});

My Question is : Could I randomize the choices array without looping through all documents as now I am doing?

Answer 1

shuffle = function(v){
    //+ Jonas Raoni Soares Silva
    //@ http://jsfromhell.com/array/shuffle [rev. #1]

    for(var j, x, i = v.length; i; j = parseInt(Math.random() * i), x = v[--i], v[i] = v[j], v[j] = x);
    return v;
};

app.get('/api/companies/:companyId/quizzes', function(req, res){

    var Query = Quiz.find({company:req.params.companyId});
    Query.populate('questions');
    Query.exec(function(err, docs){
    
    var raw = docs.toObject();
    
    //shuffle choices
    raw.questions.map(el => shuffle(el.choices))
    
    //if you need to shuffle the questions too
    shuffle(raw.questions);
    
    //if you need to limit the output questions, especially when ouput questions needs to be a subset of a pool of questions
    raw.questions.splice(limit);

        res.json(raw); // output quiz with shuffled questions and answers
    });

});

Answer 2

The essence of the question comes down to "Can I randomly shuffle results and have MongoDB do the work for me?". Well yes you can, but the important thing to remember here is that "populate" is not longer going to be your friend in helping you do so and you will need to perform the work that is doing yourself.

The short part of this is we are going to "hand-off" your client side "shuffle" to mapReduce in order to process the shuffling of the "choices" on the server. Just for kicks, I'm adding in a technique to shuffle your "questions" as well:

  var Query = Quiz.findOne({ company: "5382a58bb7ea27c9301aa9df" });
  Query.populate('company', 'name -_id');
  Query.exec(function(err,quiz) {

    var shuffle = function(v) {
      for(var j, x, i = v.length; i; j = parseInt(Math.random() * i), x = v[--i], v[i] = v[j], v[j] = x);
    };


    if (err)
      throw err;

    var raw = quiz.toObject();

    shuffle( raw.questions );

    Question.mapReduce(
      {
        map: function() {

          shuffle( this.choices );

          var found = -1;
          for ( var n=0; n<inputs.length; n++ ) {
            if ( this._id.toString() == inputs[n].toString() ) {
              found = n;
              break;
            }
          }

          emit( found, this );

        },

        reduce: function() {},

        scope: { inputs: raw.questions, shuffle: shuffle },

        query: { "_id": { "$in": raw.questions } }

      },
      function(err,results) {
        if (err)
          throw err;

        raw.questions = results.map(function(x) {
          return x.value;
        });

        console.log( JSON.stringify( raw, undefined, 4 ) );
      }
    );

  });

So the essential part of this is rather than allowing "populate" to pull all the related question information into your schema object, you are doing a manual replacement using mapReduce.

Note that the "schema document" must be converted to a plain object which is done by the .toObject() call in there in order to allow us to replace "questions" with something that would not match the schema type.

We give mapReduce a query to select the required questions from the model by simply passing in the "questions" array as an argument to match on _id. Really nothing directly different to what "populate" does for you behind the scenes, it's just that we are going to handle the "merge" manually.

The "shuffle" function is now executed on the server, which since it was declared as a var we can easily pass in via the "scope", and the "options" array will be shuffled before it is emitted, and eventually returned.

The other optional as I said was that we are also "shuffling" the questions, which is merely done by calling "shuffle" on just the _id values of the "questions" array and then passing this into the "scope". Noting that this is also passed to the query via $in but that alone does not guarantee the return order.

The trick employed here is that mapReduce at the "map" stage, must "emit" all keys in their ascending order to later stages. So by comparing the current _id value to where it's position is as an index value of the "inputs" array from scope then there is a positional order that can be emitted as the "key" value here to respect the order of the shuffle done already.

The "merging" then is quite simple as we just replace the "questions" array with the values returned from the mapReduce. There is a little help here from the .map() Array function here to clean up the results from the way mapReduce returns things.

Aside from the fact that your "options" are now actually shuffled on the server rather than through a loop, this should give you ideas of how to "custom populate" for other functions such as "slicing" and "paging" the array of referenced "questions" if that is something else you might want to look at.