CODEWITHSUNDEEP
ruby-on-railsrubygoogle-mapsgeocoding
user2923767
user2923767

Reputation: 637

Geocoder ruby gem on demand?

I have a large database with 4+ million addresses/records. The rake command (below) worked fine when the database was a small test set, but now with the large database it just simply stalls.

rake geocode:all CLASS=YourModel

2 questions: 1. Is there any simple method to have geocoder code a null/nil lat and long when the records are called (on the fly). I have a feeling that this would be hard. 2. Anyone else have problems with geocode-ing a large dataset and using the rake command?

Thanks!

Upvotes: 1

Views: 374

Answers (2)

Roman Kiselenko
Roman Kiselenko

Reputation: 44370

Update:

I create pull request based on this answer and now you can use batch in geocoder:

rake geocode:all CLASS=YourModel SLEEP=0.25 BATCH=100

I would use this solution for large database, i take it from geocoder gem rake task: You can refine this for your needs. Some example create rake task:

namespace :geocode_my_data do
  desc "Geocode all objects in my databse."
  task all: :environment do
  klass = User
    klass.where(geocoded: false).find_each(limit: 100) do |obj|
      obj.geocode; obj.save
    end
  end
end

$> rake geocode_my_data:all

Upvotes: 2

user2923767
user2923767

Reputation: 637

Used below code and put it into my lib/tasks folder as geocode_my_data.rake To run:

rake geocode:all CLASS=YourModel

Works great!

namespace :geocode_my_data do
  desc "Geocode all objects without coordinates."
  task :all => :environment do
    class_name = ENV['CLASS'] || ENV['class']
    sleep_timer = ENV['SLEEP'] || ENV['sleep']
    raise "Please specify a CLASS (model)" unless class_name
    klass = class_from_string(class_name)

klass.not_geocoded.find_each(batch_size: 100) do |obj|
      obj.geocode; obj.save
      sleep(sleep_timer.to_f) unless sleep_timer.nil?
    end
  end
end

##
# Get a class object from the string given in the shell environment.
# Similar to ActiveSupport's +constantize+ method.
#
def class_from_string(class_name)
  parts = class_name.split("::")
  constant = Object
  parts.each do |part|
    constant = constant.const_get(part)
  end
  constant
end

Upvotes: 0

Related Questions