CODEWITHSUNDEEP
ruby-on-railspostgresqlelasticsearch
lessless
lessless

Reputation: 896

RoR: Migrating from PostgreSQL to Elastic

we are looking to transfer a part of our database to from PostgreSQL to Elastic: basically we want to combine three tables - Properties, Listings and Addresses into single document.

I couldn't find any standard tools and probably, as we have a Ruby on Rails application, the pimpliest and most reliable way will be to write a migration which will just iterate through the models, compose them into single document and save to Elastic.

The task doesn't seem to be complicated but as that it's my first experience with Elastic I want to check with the community.

Thanks.

Upvotes: 0

Views: 154

Answers (1)

Paul Sanwald
Paul Sanwald

Reputation: 11329

The closest thing I'm away of is the JDBC importer. However, I think writing your own script is probably equally as fast.

There is a postgres function, row_to_json, that will convert a resulting row to JSON, which you can then publish into elasticsearch. There's nothing I'm aware of that will automatically do this for you. Assuming it's not billions of rows, I'd stick with your plan of writing a short script to run your query, and HTTP post the results into elasticsearch.

You'll need to decide on two things: Index name, and document type(s).

Some notes:

  • The consistency model between a relational database like postgres and an eventually consistent document store like elasticsearch are quite different. You should be aware of these differences and the drawbacks of them.
  • You will likely want the data in elasticsearch to be de-normalized, as there are no awesome ways of doing joins.

Upvotes: 1

Related Questions