belongsTo only being set on first and last member of hasMany

Question

My adapter uses findHasMany to load child records for a hasMany relationship. My findHasMany adapter method is directly based on the test case for findHasMany. It retrieves the contents of the hasMany on demand, and eventually does the following two operations:

store.loadMany(type, hashes);
// ...
store.loadHasMany(record, relationship.key, ids);

(The full code for the findHasMany is below, in case the issue is there, but I don't think so.)

The really strange behavior is: it seems that somewhere within loadHasMany (or in some subsequent async process) only the first and last child records get their inverse belongsTo property set, even though all the child records are added to the hasMany side. I.e., if posts/1 has 10 comments, this is what I get, after everything has loaded:

var post = App.Posts.find('1');
post.get('comments').objectAt(0).get('post'); // <App.Post:ember123:1>
post.get('comments').objectAt(1).get('post'); // null
post.get('comments').objectAt(2).get('post'); // null
// ...
post.get('comments').objectAt(8).get('post'); // null
post.get('comments').objectAt(9).get('post'); // <App.Post:ember123:1>

My adapter is a subclass of DS.RESTAdapter, and I don't think I'm overloading anything in my adapter or serializer that would cause this behavior.

Has anybody seen something like this before? It's weird enough I though someone might know why it's happening.

Extra

Using findHasMany lets me load the contents of the hasMany only when the property is accessed (valuable in my case because calculating the array of IDs would be expensive). So say I have the classic posts/comments example models, the server returns for posts/1:

{
  post: {
    id: 1,
    text: "Linkbait!"
    comments: "/posts/1/comments"
  }
}

Then my adapter can retrieve /posts/1/comments on demand, which looks like this:

{
  comments: [
    {
      id: 201,
      text: "Nuh uh"
    },
    {
      id: 202,
      text: "Yeah huh"
    },
    {
      id: 203,
      text: "Nazi Germany"
    }
  ]
}

Here is the code for the findHasMany method in my adapter:

findHasMany: function(store, record, relationship, details) {
    var type = relationship.type;
    var root = this.rootForType(type);
    var url = (typeof(details) == 'string' || details instanceof String) ? details : this.buildURL(root);
    var query = relationship.options.query ? relationship.options.query(record) : {};

    this.ajax(url, "GET", {
        data: query,
        success: function(json) {
            var serializer = this.get('serializer');
            var pluralRoot = serializer.pluralize(root);
            var hashes = json[pluralRoot]; //FIXME: Should call some serializer method to get this?

            store.loadMany(type, hashes);

            // add ids to record...
            var ids = [];
            var len = hashes.length;
            for(var i = 0; i < len; i++){
                ids.push(serializer.extractId(type, hashes[i]));
            }
            store.loadHasMany(record, relationship.key, ids);
        }
    });
}

Answer 1

Solution

Override the DS.RelationshipChange.getByReference method by inserting the following code into your app:

DS.RelationshipChange.prototype.getByReference = function(reference) {
  var store = this.store;

  // return null or undefined if the original reference was null or undefined
  if (!reference) { return reference; }

  if (reference.record) {
    return reference.record;
  }

  return store.materializeRecord(reference);
};

Yes, this is overriding a private, internal method in Ember Data. Yes, it may break at any time with any update. I'm pretty sure this is a bug in Ember Data, but I'm not 100% certain this is the right solution. But it does solve this problem, and possibly other relationship-related problems.

This fix is designed to be applied to Ember Data master as of 29 Apr 2013.

Reason

DS.Store.loadHasMany calls DS.Model.hasManyDidChange, which retrieves references for all the child records and then sets the hasMany's content to the array of references. This kicks off a chain of observers., eventually calling DS.ManyArray.arrayContentDidChange, in which the first line is this._super.apply(this, arguments);, calling the superclass method Ember.Array.arrayContentDidChange. That Ember.Array method includes an optimization that caches the first and last object in the array and calls objectAt on only those two array members. So there's the part that singles out the first and last record.

Next, since DS.RecordArray implements an objectAtContent method (from Ember.ArrayProxy), the objectAtContent implementation calls DS.Store.recordForReference, which in turn calls DS.Store.materializeRecord. This last function adds a record property to the reference that is passed in as a side effect.

Now we get to what I think is a bug. In DS.ManyArray.arrayContentDidChange, after calling the superclass method, it loops through all the new references and creates a DS.RelationshipChangeAdd instance that encapsulates the owner and child record references. But the first line inside the loop is:

var reference = get(this, 'content').objectAt(i);

Unlike what happens above to the first and last record, this calls objectAt directly on the Ember.NativeArray and bypasses the ArrayProxy methods including the objectAtContent hook, which means that DS.Store.materializeRecord--which adds the record property on the reference object--may have never been called on some references.

Next, the relationship changes created in the loop are immediately afterward (in the same run loop) applied with this call tree: DS.RelationshipChangeAdd.sync -> DS.RelationshipChange.getFirstRecord -> DS.RelationshipChange.getByReference. This last method expects the reference object to have a record property. However, the record property is only set on the first and last reference objects, for reasons explained above. Therefore, for all but the first and last records, the relationship fails to be established because it doesn't have access to the child record object!

The above fix calls DS.Store.materializeRecord whenever the record property doesn't exist on the reference. The last line in the function is the only thing added. On the one hand, it looks like this was the original intention: that var store = this.store; line in the original declares a variable that isn't otherwise used in the function, so what's it there for? Also, without the added line, the function doesn't always return a value, which is a little unusual for a function which is expected to do so. On the other hand, this could lead to mass materialization in some cases where that would be undesirable (but, the relationships just won't work without it in some cases, it seems).

Possibly related

The "chain of observers" I mentioned takes a bit of an odd path. The initiating event was setting the content property on a DS.ManyArray, which extends Ember.ArrayProxy--therefore the content property has a dependent property arrangedContent. Importantly, the observers on arrangedContent are executed before observers on content are executed (see Ember.propertyDidChange). However, the default implementation of Ember.ArrayProxy.arrangedContentArrayDidChange simply calls Ember.Array.arrayContentDidChange, which DS.ManyArray implements! The point being, this looks like a recipe for some code to execute in an unintended order. That is, I think Ember.ManyArray.arrayContentDidChange may getting executed earlier than expected. If this is the case, the above mentioned code that expects the record property to already exist on all references may have been expecting this reasonably, as one of the observers directly on the content property may call DS.Store.materializeRecord on each reference. But I haven't dug deep enough to find out if this is true.