Rails/ActiveRecord has_many through: association on unsaved objects

Question

Let's work with these classes:

class User < ActiveRecord::Base
    has_many :project_participations
    has_many :projects, through: :project_participations, inverse_of: :users
end

class ProjectParticipation < ActiveRecord::Base
    belongs_to :user
    belongs_to :project

    enum role: { member: 0, manager: 1 }
end

class Project < ActiveRecord::Base
    has_many :project_participations
    has_many :users, through: :project_participations, inverse_of: :projects
end

A user can participate in many projects with a role as a member or a manager. The connecting model is called ProjectParticipation.

I now have a problem using the associations on unsaved objects. The following commands work like I think they should work:

# first example

u = User.new
p = Project.new

u.projects << p

u.projects
=> #<ActiveRecord::Associations::CollectionProxy [#<Project id: nil>]>

u.project_participations
=> #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil>]>

So far so good - AR created the ProjectParticipation by itself and I can access the projects of a user with u.projects.

But it does not work if I create the ProjectParticipation by myself:

# second example

u = User.new
pp = ProjectParticipation.new
p = Project.new

pp.project = p # assign project to project_participation

u.project_participations << pp # assign project_participation to user

u.project_participations
=> #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil>]>

u.projects
=> #<ActiveRecord::Associations::CollectionProxy []>

Why are the projects empty? I cannot access the projects by u.projects like before.

But if I go through the participations directly, the project shows up:

u.project_participations.map(&:project)
=> [#<Project id: nil>]

Shouldn't it work like the first example directly: u.projects returning me all projects not depending on whether I create the join object by myself or not? Or how can I make AR aware of this?

Answer 1

At the risk of some serious oversimplification let me try to explain what is going on

What Most of the other answers are trying to tell you is that these objects have not been linked yet by active record until they are persisted in the DB. Consequently the association behavior that you are expecting is not fully wired up.

Notice that this line from your first example

 u.project_participations
 => #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil>]>

Is identical to the result from your second example

u.project_participations
=> #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil,   project_id: nil, role: nil>]>

This statement from your analysis of what you think rails is doing is inaccurate:

So far so good - AR created the ProjectParticipation by itself and I can access the projects of a user with u.projects.

AR record has not created the ProjectParticipation. You have declared this relationship in your model. AR is just returning proxy for the collection that it will have at some point in the future, which when populated assigned, etc, you will be be able to iterate over and query its members etc.

The reason that this works:

u.projects << p

u.projects
=> #<ActiveRecord::Associations::CollectionProxy [#<Project id: nil>]>

But this doesn't

pp.project = p # assign project to project_participation

u.project_participations << pp # assign project_participation to user

u.project_participations
=> #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil>]>

u.projects
=> #<ActiveRecord::Associations::CollectionProxy []>

Is that in the first case you are just adding objects to an array that your user instance has direct access to. In the second example the has_many_through relationship reflects a relationship that happens at the database level. In the second example in order for the your projects to be accessible through your user, AR has to actually run a query that joins the tables and returns the data you are looking for. Since none of these objects is persisted yet that database query can't happen yet so all you get back are the proxies.

The last bit of code is misleading because it is not actually doing what you think.

u.project_participations.map(&:project)
=> [#<Project id: nil>]

In this case you have a user which is directly holding an array of ProjectParticipations one of which is directly holding a project so it works. It is not actually using the has_many_through mechanism in they way you think.

Again this is a bit of an oversimplification but that is the general idea.

Answer 2

Short answer: No, second example won't work the way it worked in first example. You must use first example's way of creating intermediate associations directly with user and project objects.

Long answer:

Before we start, we should know how has_many :through is being handled in ActiveRecord::Base. So, let's start with has_many(name, scope = nil, options = {}, &extension) method which calls its association builder here, at the end of method the returned reflection and then add reflection to a hash as a cache with key-value pair here.

Now question is, how do these associations gets activated?!?!

It's because of association(name) method. Which calls association_class method, which actually calls and return this constant: Associations::HasManyThroughAssociation, that makes this line to autoload active_record/associations/has_many_through_association.rb and instantiate its instance here. This is where owner and reflection are saved when the association is being created and in the next reset method is being called which gets invoked in the subclass ActiveRecord::Associations::CollectionAssociation here.

Why this reset call was important? Because, it sets @target as an array. This @target is the array where all associated objects are stored when you make a query and then used as cache when you reuse it in your code instead of making a new query. That's why calling user.projects(where user and projects persists in db, i.e. calling: user = User.find(1) and then user.projects) will make a db query and calling it again won't.

So, when you make a reader call on an association, e.g.: user.projects, it invokes the collectionProxy, before populating the @target from load_target.

This is barely scratching the surface. But, you get the idea how associations are being build using builders(which creates different reflection based on the condition) and creates proxies for reading data in the target variable.

tl;dr

The difference between your first and second examples is the way their association builders are being invoked for creating associations' reflection(based on macro), proxy and target instance variables.

First example:

u = User.new
p = Project.new
u.projects << p

u.association(:projects)
#=> ActiveRecord::Associations::HasManyThroughAssociation object
#=> @proxy = #<ActiveRecord::Associations::CollectionProxy [#<Project id: nil, name: nil, created_at: nil, updated_at: nil>]>
#=> @target = [#<Project id: nil, name: nil, created_at: nil, updated_at: nil>]

u.association(:project_participations)
#=> ActiveRecord::Associations::HasManyAssociation object
#=> @proxy = #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil, created_at: nil, updated_at: nil>]>
#=> @target = [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil, created_at: nil, updated_at: nil>]

u.project_participations.first.association(:project)
#=> ActiveRecord::Associations::BelongsToAssociation object
#=> @target = #<Project id: nil, name: nil, created_at: nil, updated_at: nil>

Second example:

u = User.new
pp = ProjectParticipation.new
p = Project.new

pp.project = p # assign project to project_participation

u.project_participations << pp # assign project_participation to user

u.association(:projects)
#=> ActiveRecord::Associations::HasManyThroughAssociation object
#=> @proxy = nil
#=> @target = []

u.association(:project_participations)
#=> ActiveRecord::Associations::HasManyAssociation object
#=> @proxy = #<ActiveRecord::Associations::CollectionProxy [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil, created_at: nil, updated_at: nil>
#=> @target = [#<ProjectParticipation id: nil, user_id: nil, project_id: nil, role: nil, created_at: nil, updated_at: nil>]

u.project_participations.first.association(:project)
#=> ActiveRecord::Associations::BelongsToAssociation object
#=> @target = #<Project id: nil, name: nil, created_at: nil, updated_at: nil>

There's no proxy for BelongsToAssociation, it has just target and owner.

However, if you are really inclined to make your second example work, you will just have to do this:

u.association(:projects).instance_variable_set('@target', [p])

And now:

u.projects
#=>  #<ActiveRecord::Associations::CollectionProxy [#<Project id: nil, name: nil, created_at: nil, updated_at: nil>]>

In my opinion which is a very bad way of creating/saving associations. So, stick with the first example itself.

Answer 3

This is more of a rails structure thing at the level of the ruby data structures. To simplify it lets put it this way. First of all imagine User as a data structure contains:

1. project_participations Array
2. projects Array

And Project

1. users Array
2. project_participations Array

Now when you mark a relation to be :through another (user.projects through user.project_participations)

Rails implies that when you add a record to that first relation (user.projects) it will have to create another one in the second realation (user.project_participations) that is all the effect of the 'through' hook

So in this case,

user.projects << project
#will proc the 'through'
#user.project_participations << new_entry

Keep in mind that the project.users is still not updated because its a completely different data structure and you have no reference to it.

So lets take a look what will happen with the second example

u.project_participations << pp
#this has nothing hooked to it so it operates like a normal array

So In conclusion, this acts like a one way binding on a ruby data structure level and whenever you save and refresh your objects, this will behave the way you wanted.

Answer 4

Associations are defined on database level and make use of database table's primary key (and in polymorphic cases, class name). In case of has_many :through the lookup on association (say, User's Projects) is:

1. Fetch all User-Project pairs, whose user_id is a certain value (primary key of an existing User in the database)
2. Fetch all project_id (primary keys of projects) from these pairs
3. Fetch all Projects by resulting keys

Of course, these are simple terms, in database terms it's much shorter and uses more complicated abstractions, such as an inner join, but the essence is the same.

When you create a new object via new, it is not yet saved in the database, and therefore has no primary key (it's nil). That said, if the object is not in a database yet, you have no way of referencing it from any ActiveRecord's association.

Side note:
There is a possibility, however, that a newly created (and not saved yet) object will act as if something is associated with it: it might show entries belonging to NULL. This usually means you have an error in your database schema that allows such things to happen, but hypothetically, one could design his database to make use of this.