CODEWITHSUNDEEP
ruby-on-railsrubyfriendly-id
viotech
viotech

Reputation: 105

Rails 4.1 - Saving friendly_id slug for many records en masse

I have a database with a few million entities needing a friendly_id slug. Is there a way to speed up the process of saving the entities? find_each(&:save) is very slow. At 6-10 per second I'm looking at over a week of running this 24/7.

I'm just wondering if there is a method within friendly_id or parallel processing trick that can speed this process up drastically.

Currently I'm running about 10 consoles, and within each console starting the value +100k:

Model.where(slug: nil).find_each(start: value) do |e|
  puts e.id
  e.save
end

EDIT

Well one of the biggest things that was causing the updates to go so insanely slow is the initial find query of the entity, and not the actual saving of the record. I put the site live the other day, and looking at server database requests continually hitting 2000ms and the culprit was @entity = Entity.find(params[:id]) causing the most problems with 5+ million records. I didn't realize there was no index on the slug column and active record is doing its SELECT statements on the slug column. After indexing properly, I get 20ms response times and running the above query went from 1-2 entities per second to 1k per second. Doing multiple of them got the job down quick enough for the one time operation.

Upvotes: 0

Views: 471

Answers (1)

Danielspector
Danielspector

Reputation: 58

I think the fastest way to do this would be to go straight to the database, rather than using Active Record. If you have a GUI like Sequel Pro, connect to your database (the details are in your database.yml) and execute a query there. If you're comfortable on the command line you can run it straight in the database console window. Ruby and Active Record will just slow you down for something like this.

To update all the slugs of a hypothetical table called "users" where the slug will be a concatenation of their first name and last name you could do something like this in MySQL:

UPDATE users SET slug = CONCAT(first_name, "-", last_name) WHERE slug IS NULL

Upvotes: 2

Related Questions