CODEWITHSUNDEEP
ruby-on-railsrubymongodbmongoidnosql
cortex
cortex

Reputation: 5206

Mongoid has_many relation too slow (never ends) - 40k documents

I have a Client and a Group models. A client has many groups.

class Client
  include Mongoid::Document
  include Mongoid::Paranoia
  include Mongoid::Timestamps

  field ...

  has_many :groups, dependent: :delete

  ...
end

class Group
  ...

  belongs_to :client
end

One client has 40k groups and when I try client.groups it takes too long. I have wait for several minutes and it never ends.

MONGODB (14.2ms) humtl_development['groups'].find({:deleted_at=>nil, "client_id"=>BSON::ObjectId('51b37763218c5a19e0000048')})
MONGODB [DEBUG] cursor.refresh() for cursor 48594378191047181

I found this in mongoid docs.

Are 40k too much documents? Should I get rid of has_many relation and use embeds_many instead? The problem is my mongoid version? MongoDB configuration? Any advice is appreciated.

For sure I don't need to display all 40k groups, what I need is User.where(:group_id.in => client.groups.map(&:id)).

Thanks so much.

PS: MongoDB v2.4.3 mongoid v2.7.1 mongo v1.9.2

Upvotes: 2

Views: 1249

Answers (1)

Iuri G.
Iuri G.

Reputation: 10630

Make sure you have client_id indexed on your document so lookups are fast. then you can use distinct(:_id) to get 40K ids in an array. so your final query will look like: User.where(:group_id.in => client.groups.distinct(:_id)). That is the most efficient way I can think of to get array of 40K ids. Try that - it might work, but as everyone stated already - it's not webscale ;)

From what I can see, you are trying to get all users for a given client, and your user might be associated to multiple clients through different groups. In order to be able to efficiently look up Users for clients without join (since you dont have join in mongoid), you can use HABTM on users with clients without inverse relation. So I would do following:

class Client
  include Mongoid::Document
  include Mongoid::Paranoia
  include Mongoid::Timestamps

  field ...

  has_and_belongs_to_many :users
  has_many :groups, dependent: :delete

  ...
end

class User
  has_and_belongs_to_many :clients
end

Then every time you associate user to a group, remember to add group's client to user (user.clients << client or client.users << user) so they know about each other. Don't worry about duplication since mongoid is using Set so if relation between user and client already exists, its not going to do anything. Then you can look up users from client: client.users or lookup users for multiple clients: User.where(:client_ids.in => client_ids)

Upvotes: 2

Related Questions