CODEWITHSUNDEEP
ruby-on-railsrubyactiverecord
dc10
dc10

Reputation: 2198

How can I bulk save ActiveRecord Objects with validation in a fast way?

I want to save 1000 ActiveRecord Models in a fast way. I know about several options e.b. batch import gem or native insert with sql. But the problem is that these solutions do not check whether a record already exists. Before a record gets saved I definitely need to check before whether this record already exists by checking some attributes. Currently I iterate and do save like this

rate = Rate.find_or_create_by(
            :supplier_id => acc.supplier_id,
            :candidate_id => acc.id,
            :total => self.total,
            :catering => self.catering,
            :arrival => arrival,
            :departure => departure

        ) do |r|
          r.arrival = arrival
          r.departure = departure
          r.supplier_id = acc.supplier_id
          r.total = self.total
          r.currency = currency
        end

How is this possible in a fast way with a lot of objects?

Upvotes: 1

Views: 1749

Answers (1)

ABMagil
ABMagil

Reputation: 5635

find_or_create, while a single method, actually performs two DB queries. Not only is this slow, it's not even a guarantee of uniqueness (just a very high likelihood). The same is true of validates :uniqueness.

The fastest way to do what you want (with a guarantee of uniqueness) would probably be to build a UNIQUE constraint or index in the database. Then, on save, if the insert fails, Rails will raise an exception. That will prevent duplicate records, as well as being as fast as standard rails.

If you want speed improvements over the "Rails way" (looping through records, as you're already doing), you'll need to get a bit tricky. From this article, the fastest way to do it is to build a single insert with all your records in raw SQL, then run that statement. Of course, that's predicated on guaranteeing within rails that you're not creating any duplicate records.

Upvotes: 2

Related Questions