CODEWITHSUNDEEP
ruby-on-railsrubypostgresqlrake-task
Abdul Baig
Abdul Baig

Reputation: 3721

Update huge data in rake task

i have a model named PageDensity which has about 5m rows.

When i created project PageDensity table stored float of 5 decimal precision in density coulmn.

Now the requirement changed to round it to two decimal places.

I simply wrote a task to round all the densities but it makes system so heavy that is stucks. Even i can't use query as my rounding is bit change like 0.57500 will be rounded to 0.57 and 0.57600 will be rounded to 0.58.

what i have tried so far is simply:

  task round_densities: :environment do
    application_object = ApplicationController.new
    time = Benchmark.realtime do
      ActiveRecord::Base.transaction do
        PageDensity.all.each {|p| p.update_attributes(density: application_object.round_number(p.density))}
      end
    end
    puts '***************************************'
    puts "Total Time Consumed #{time} seconds"
    puts '***************************************'
  end

and i also tried to make query for rounding but failed:

select round(0.00500, 2)
#this returns 0.01 as this should return 0.00

I am using postgres any idea to make it psql query or by using rails ?

Upvotes: 3

Views: 699

Answers (2)

Shadwell
Shadwell

Reputation: 34774

It sounds like your rounding requirement is just 0.001 off what the normal rounding would be.

In which case I think you can run the sql update:

update page_densities set density = round(density - 0.001, 2)

This will round like this:

0.011 => round(0.010, 2) => 0.01
0.015 => round(0.014, 2) => 0.01
0.016 => round(0.015, 2) => 0.02
0.02  => round(0.019, 2) => 0.02

Upvotes: 2

apneadiving
apneadiving

Reputation: 115521

You should work with batches so change

PageDensity.all.each

with:

PageDensity.all.find_each

Check doc.

But to do so, you'd have to remove the transaction which is actually useless.

Sideremark, if you have no callback to trigger, replace:

p.update_attributes(density: application_object.round_number(p.density))

with:

p.update_column(:density, application_object.round_number(p.density))

This will save some additional time.

Upvotes: 2

Related Questions