Append inside Schemaless Array

Question

I am new to MongoDB, so far playing around with it, confronted with a problem, here i am having a hard time when trying to append multiple objects inside Schema-Less Array.So far i tried $push to append multiple objects inside array but got a Mongo Error.

[MongoError: Can't use $push/$pushALL within non-array

i don't know why i am getting this error, when using $push with array

Schema:

EventTypeSchema = new Schema(){
type: String,
eventID: {
type: Schema.Types.ObjectId,
ref: 'User'
}
} 

PersonSchema = new Schema(){
PersonID: {
type: Schema.Types.ObjectId,
ref: 'User'
}
Invitation: [  ]    //Schema-less
}

In Controller i have Access to both EventType and Person Model Controller:

exports.update = function(req,res){
var event = new EventType();
event.type = 'EVENT';
event.eventID = req.body.eventid;
var query = {'PersonID': req.body.personid};
var update = {$push:{'Invitation': event}};
Person.update(query,update,function(err,user){...})
};

for debugging purposes i tried to give Mixed type Schema for Array but didn't get it to work

PersonSchema = new Schema(){
PersonID: {
type: Schema.Types.ObjectId,
ref: 'User'
}
Invitation: [ {
type: Schema.Types.Mixed
} ]    
}       

When i removed $push on update then only whole event object is getting inside Invitation, the reason i created Schema-less array is because i am dealing with different type of invitation, here i just described about event invitation, otherwise there are different type of invitations i am dealing with like, User Invitation for request, Conference invitation, so there would combination of different objectId's, i think there should be the way to append to schema-less array in mongoDB.

EDIT:

The following is what I came up with. Not able to get it to work though.

function PortalTypes() {
    Schema.apply(this,arguments);
    this.add({
        object_type: String,
    });
}

util.inherits( PortalTypes, Schema );

var userType = new PortalTypes({
    ID : {
        type: Schema.Types.ObjectId,
        ref : 'User'
    }
});

var eventType = new PortalTypes({
    ID : {
        type: Schema.Types.ObjectId,
        ref : 'events'
    }
});



var user = new userType({ID:'dsaj3232--objectID','object_type':'user'});
user.save();

var event = new eventType({ID:'dasddehiqe98--objectID','object_type':'event'});
event.save();

Networks.Invitation.push(user,event);

How can I do something like this?

Answer 1

Despite your schema that error at the top means that that there is a matching document in the collection that does not have this field set as an array, but it's present with another type. Possibly just a string or object.

Here's a little, contrived example listing to demonstrate:

var async = require('async'),
    mongoose = require('mongoose'),
    Schema = mongoose.Schema;

var personSchema = new Schema({
  invitation: []
});

var Person = mongoose.model( 'Person', personSchema );

mongoose.connect('mongodb://localhost/test');

async.waterfall(
  [
    function(callback) {
      Person.remove({},function(err,num) {
        callback(err);
      });
    },

    function(callback) {
      console.log( "Creating" );
      var person = new Person();
      person.save(function(err,person) {
        if (err) callback(err);
        console.log(person);
        callback(err,person);
      });
    },

    function(person,callback) {
      console.log( "Updating" );
      Person.findOneAndUpdate(
        { "_id": person._id },
        { "$push": { "invitation": "something" } },
        function(err,doc) {
          if (err) callback(err);
          console.log(doc);
          callback(err);
        }
      );
    },

    function(callback) {
      console.log( "Upserting" );
      Person.findOneAndUpdate(
        { "name": "bob" },
        { "$set": { "invitation": {} } },
        { "upsert": true },
        function(err,doc) {
          if(err) callback(err);
          console.log(doc);
          callback(err,doc);
        }
      );
    },

    function(bob,callback) {
      console.log( "Failing" );
      Person.findOneAndUpdate(
        { "name": "bob" },
        { "$push": { "invitation": "else" } },
        function(err,doc) {
          if (err) callback(err);
          console.log(doc);
          callback(err);
        }
      );
    }

  ],
  function(err) {
    if (err) throw err;
    console.log( "Done" );
    mongoose.disconnect();

  }
);

That should give results like this:

Creating
{ __v: 0, _id: 54a18afb345b4efc02f21020, invitation: [] }
Updating
{ _id: 54a18afb345b4efc02f21020,
  __v: 0,
  invitation: [ 'something' ] }
Upserting
{ _id: 54a18afb9997ca0c4a7eb722,
  name: 'bob',
  __v: 0,
  invitation: [ {} ] }
Failing

/home/neillunn/scratch/persons/node_modules/mongoose/lib/utils.js:413
    throw err;
          ^
MongoError: exception: The field 'invitation' must be an array but is of type Object
in document   {_id: ObjectId('54a18afb9997ca0c4a7eb722')}

The error message is a bit different since they were improved a bit in MongoDB 2.6 and upwards ( where this error string comes from ) to be a bit more precise about the actual problem. So in modern versions you would be told exactly what was wrong.

Despite the schema, methods like .update() ( I used .findOneAndUpdate() for convenience ) bypass the mongoose schema definition somewhat and go right to the database. So it's possible to do this and also possible you just had a document in place already, or otherwise created when a different schema definition was in place.

So that's the first problem here.

---

The rest of what you seem to be asking is for a "polymorphic" type of association in the array, and also where you do not wish to "embed" the whole created object in the array but just a reference to it.

Mongoose has "discriminators" to allow for this sort of thing, allowing different model types for objects to be stored within the same collection, but resolving to their own object and schema "type".

Following the current documentation example, here is an example listing of what that might look like:

var util = require('util'),
    async = require('async'),
    mongoose = require('mongoose'),
    Schema = mongoose.Schema;

function logger(label,content) {
  console.log(
    "%s:\n%s\n", label, JSON.stringify( content, undefined, 4 ) );
}

function BaseSchema() {

  Schema.apply(this,arguments);

  this.add({
    name: String,
    createdAt: { type: Date, default: Date.now }
  });

}

util.inherits( BaseSchema, Schema );


var personSchema = new BaseSchema(),
    bossSchema = new BaseSchema({ department: String });

var companySchema = new Schema({
  people: [{ type: Schema.Types.ObjectId, ref: 'Person' }]
});


var Person = mongoose.model( 'Person', personSchema ),
    Boss = Person.discriminator( 'Boss', bossSchema ),
    Company = mongoose.model( 'Company', companySchema );

mongoose.connect('mongodb://localhost/test');

async.waterfall(
  [
    function(callback) {
      Company.remove({},function(err,num) {
        callback(err);
      });
    },
    function(callback) {
      Person.remove({},function(err,num) {
        callback(err);
      });
    },
    function(callback) {
      var person = new Person({ name: "Bob" });
      person.save(function(err,person) {
        logger("Person", person);
        callback(err,person);
      });
    },
    function(person,callback) {
      var boss = new Boss({ name: "Ted", department: "Accounts" });
      boss.save(function(err,boss) {
        logger("Boss", boss);
        callback(err,person,boss);
      });
    },
    function(person,boss,callback) {
      var company = new Company();
      company.people.push(person,boss);
      company.save(function(err,company) {
        logger("Stored",company);
        callback(err,company);
      });
    },
    function(company,callback) {
      Company.findById(company.id)
        .populate('people')
        .exec(function(err,company) {
          logger("Polulated",company);
          callback(err);
        });
    }
  ],
  function(err) {
    if (err) throw err;
    mongoose.disconnect();
  }
);

Which will produce output like this:

Person:
{
    "__v": 0,
    "name": "Bob",
    "createdAt": "2014-12-29T17:53:22.418Z",
    "_id": "54a1951210a7a1b603161119"
}

Boss:
{
    "__v": 0,
    "name": "Ted",
    "department": "Accounts",
    "__t": "Boss",
    "createdAt": "2014-12-29T17:53:22.439Z",
    "_id": "54a1951210a7a1b60316111a"
}

Stored:
{
    "__v": 0,
    "_id": "54a1951210a7a1b60316111b",
    "people": [
        "54a1951210a7a1b603161119",
        "54a1951210a7a1b60316111a"
    ]
}

Polulated:
{
    "_id": "54a1951210a7a1b60316111b",
    "__v": 0,
    "people": [
        {
            "_id": "54a1951210a7a1b603161119",
            "name": "Bob",
            "__v": 0,
            "createdAt": "2014-12-29T17:53:22.418Z"
        },
        {
            "_id": "54a1951210a7a1b60316111a",
            "name": "Ted",
            "department": "Accounts",
            "__v": 0,
            "__t": "Boss",
            "createdAt": "2014-12-29T17:53:22.439Z"
        }
    ]
}

As you can see, there is a different structure for how Person and Boss are saved, notably the _t property as well as other defined properties for the different objects. Both however are actually stored in the same "people" collection and can be queried as such.

When storing these on the Company object, only the "reference id" values are stored in the array. Debatable to what you might want, but this is the difference between "referenced" and "embedded" schema models. You can see however when the .populate() method is called, then the objects are restored to their full form as they are read from the referenced collection.

---

So check your collection for existing documents that vary from your schema definition, and consider the approach as shown to represent a "polymorphic" association for different "types" of objects.

Note though that this kind of resolution is only supported under the "referenced" schema design, which can also possibly have it's drawbacks. If you want the objects stored as "embedded" within the single Company collection ( for example ), then you don't get the type of object resolution with varying schema types done by mongoose automatically. Resolving different types of objects would have to be done manually in your code, or provided plugin or however you do it.

More

Being specific to all of the purpose because there seems to be some confusion following something based on the standard documentation example, here is a more heavily commented listing:

var util = require('util'),
    async = require('async'),
    mongoose = require('mongoose'),
    Schema = mongoose.Schema;

// Utility
function logger(label,content) {
  console.log(
    "%s:\n%s\n", label,
    util.inspect( content, false, 8, false ) );
}

/*
 * Schemas:
 *
 * you can use a base schema for common fields or just a plain
 * definition
 */

var portalSchema = new Schema(),

    userSchema = new Schema({
      "name": String,
      "age": Number
    }),

    eventSchema = new Schema({
      "place": String,
      "eventDate": { type: Date, default: Date.now }
    });

/*
 * Models
 *
 * there is only one "model" defined and therefore one collection only
 * as everything is comes from a stored __v field with the "Model" name
 * defined in the discriminator
 */

var Portal = mongoose.model( 'Portal', portalSchema ),
    User = Portal.discriminator( 'User', userSchema ),
    Event = Portal.discriminator( 'Event', eventSchema );

/*
 * Then there is the thing that is going to consume the references to the
 * 'Portal' model. The array here references the "base" model.
 */

var otherSchema = new Schema({
  "afield": String,
  "portals": [{ type: Schema.Types.ObjectId, ref: "Portal" }]
});

var Other = mongoose.model( 'Other', otherSchema );

/*
 * Meat:
 *
 * Let's start doing things
 */

mongoose.connect('mongodb://localhost/test');

// Just because we're passing around objects without globals or other scoping
async.waterfall(
  [
    // Start fresh by removing all objects in the collections
    function(callback) {
      Other.remove({},function(err,num) {
        callback(err);
      });
    },
    function(callback) {
      Portal.remove({},function(err,num) {
        callback(err);
      });
    },

    // Create some portal things
    function(callback) {
      var eventObj = new Event({ "place": "here" });
      eventObj.save(function(err,eventObj) {
        logger("Event", eventObj);
        callback(err,eventObj);
      });
    },
    function(eventObj,callback) {
      var userObj = new User({ "name": "bob" });
      userObj.save(function(err,userObj) {
        logger("User", userObj);
        callback(err,eventObj,userObj);
      });
    },

    // Store the references in the array for the Other model
    function(eventObj,userObj,callback) {
      var other = new Other({
        "afield": "something"
      });
      other.portals.push(eventObj,userObj);
      other.save(function(err,other) {
        logger("Other Stored",other);
        callback(err,other);
      });
    },

    // See how it's all really stored
    function(other,callback) {
      Portal.find({},function(err,portals) {
        logger("Portals",portals);
        callback(err,other);
      });
    },

    // But watch the magic here
    function(other,callback) {
      User.find({},function(err,portals) {
        logger("Just Users!",portals);
        callback(err,other);
      });
    },


    // And constructed as one object by populate
    function(other,callback) {
      Other.findById(other.id)
        .populate('portals')
        .exec(function(err,other) {
          logger("Other populated",other);
          console.log("%s: %s",
            "1st Element", other.portals[0].constructor.modelName );
          console.log("%s: %s",
            "2nd Element", other.portals[1].constructor.modelName );

          callback(err);
        });
    }

  ],
  function(err) {
    // It's just a script, so clean up
    if (err) throw err;
    mongoose.disconnect();
  }
);

That should explain some things and what "discriminators" are. Everything is stored in just "one" collection which is bound to the base model. Everything else is defined using .discriminator() from that base. The "name" of the "class model" or "discriminator" is stored on the object. But note that is stored on the collection only, not in the place where they are referenced as that only stores the _id values. Look at the output carefully:

Event:
{ __v: 0,
  place: 'here',
  __t: 'Event',
  _id: 54a253ec456b169310d131f9,
  eventDate: Tue Dec 30 2014 18:27:40 GMT+1100 (AEDT) }

User:
{ __v: 0,
  name: 'bob',
  __t: 'User',
  _id: 54a253ec456b169310d131fa }

Other Stored:
{ __v: 0,
  afield: 'something',
  _id: 54a253ec456b169310d131fb,
  portals: [ 54a253ec456b169310d131f9, 54a253ec456b169310d131fa ] }

Portals:
[ { _id: 54a253ec456b169310d131f9,
    place: 'here',
    __v: 0,
    __t: 'Event',
    eventDate: Tue Dec 30 2014 18:27:40 GMT+1100 (AEDT) },
  { _id: 54a253ec456b169310d131fa,
    name: 'bob',
    __v: 0,
    __t: 'User' } ]

Just Users!:
[ { _id: 54a253ec456b169310d131fa,
    name: 'bob',
    __v: 0,
    __t: 'User' } ]

Other populated:
{ _id: 54a253ec456b169310d131fb,
  afield: 'something',
  __v: 0,
  portals:
   [ { _id: 54a253ec456b169310d131f9,
       place: 'here',
       __v: 0,
       __t: 'Event',
       eventDate: Tue Dec 30 2014 18:27:40 GMT+1100 (AEDT) },
     { _id: 54a253ec456b169310d131fa,
       name: 'bob',
       __v: 0,
       __t: 'User' } ] }

1st Element: Event
2nd Element: User

So there is only one collection for all "portal" types but there is some magic there as shown. The "others" collection only stores the _id values in it's array of "portals". This is how mongoose references work, where the "model" and attached schema is not stored in the data but as part of the code definition.

The "discriminator" part stores this "model name" on the field so it can be resolved to the correct type, but it's still all in the same collection, and part of the User model magic demonstrated.

Why? It's how .populate() works. Under the hood an $in operator is used with the array content, so it's all expected to be in the one place. But you can still resolve types as shown.

If you expect using separate collections, then you are doing everything manually and storing model names and querying other collections for references all by yourself.