Adding additional fields to an embedded document

Question

I make extensive use of embedded documents in my MongoDB database and I'm running into speed problems when trying to add additional data:

As an example I have a document that looks a bit like this:

    {
       "date" : <<the date>>
       "name" : "thisName"
       "basket": [
                   {
                     "stock": "IBM",
                     "quantity": 1000.0,
                     "profit"  : 10:0,
                   },
                   ...
                   {
                     "stock": "MSFT",
                     "quantity": 2000.0,
                     "profit"  : 30:0,
                   },

                 ]
}

What I want to do is to add 5 new fields in the embedded documents like this:

    {
       "date" : <<the date>>
       "name" : "thisName"
       "basket": [
                   {
                     "stock": "IBM",
                     "quantity": 1000.0,
                     "profit"  : 10:0,
                     "new_1"  : 10:0,
                     "new_2"  : 10:0,
                     "new_3"  : 10:0,
                     "new_4"  : 10:0,
                     "new_4"  : 10:0,
                     "new_5"  : 10:0
                   },
                   ...
                   {
                     "stock": "MSFT",
                     "quantity": 2000.0,
                     "profit"  : 30:0,
                     "new_1"  : 10:0,
                     "new_2"  : 10:0,
                     "new_3"  : 10:0,
                     "new_4"  : 10:0,
                     "new_4"  : 10:0,
                     "new_5"  : 10:0
                   },

                 ]
}

I started doing this using find().update_one() in a for loop, identifying each embedded document explicitly and using document explicitly using "$set". This approach works but it is very slow. If my collection was small I'm sure that this wouldn't matter but as it is it's huge (100's of millions of documents). It's probably so slow because the entire document has to be moved every time I add a set of fields. With that in mind I attempted to add the new fields to all the embedded documents in one go. I did this by leaving the find query empty and removing the positional $ from the "$set" command. A little like this (in pymongo):

bulk.find({"date": dates[i],
           "strategyId": strategyInfo[siOffset[l]][ID]
          }).update({
                     "$set": {
                          "basket.new_1": 0.0,
                          "basket.new_2": 0.0,
                          "basket.new_3": 0.0,
                          "basket.new_4": 0.0,
                          "basket.new_5": 0.0
                         }
                      })

This approach seems to throw an error cannot use the part (basket of basket.new_5) to traverse the element ({basket:......

Is anyone able to give some insight as to what I'm doing wrong? Is it even possible to do this?

Answer 1

You can use a recursive function like this.

First find all that data for update

db.collection('game_users').find(
    {"date": dates[i],"strategyId": strategyInfo[siOffset[l]][ID]}
).toArray(function(err, data) {
    var i=0;
    function data_Update(){
        if(i!=data.length){
            db.collection('game_users').update(
                {"date": dates[i],"strategyId": strategyInfo[siOffset[l]][ID]},
                { $set : {
                    "basket.new_1": 0.0,
                    "basket.new_2": 0.0,
                    "basket.new_3": 0.0,
                    "basket.new_4": 0.0,
                    "basket.new_5": 0.0
                    }
                },
                function(err, resp) {
                    i++;
                    data_Update();
                }
            );
        }
    }
}
);`