CODEWITHSUNDEEP
solr
Ram
Ram

Reputation: 55

Solr full import from oracle database

We are indexing huge data (millions of records) from oracle database to solr with very complex queries, and will be having production release every month. The problem here is that when a new column is added to database table, corresponding fields has to be added in Solr and perform a full import all millions of records where its taking 2.5 to 3 hours during prod deployment. So due to this every prod release we had to spend 3 to 4 hours of time. Is there any way where we can avoid full import rather only delta import should be performed when the newly added column in is inserted with new value, so that it will reduce our deployment time.

Upvotes: 0

Views: 112

Answers (1)

Persimmonium
Persimmonium

Reputation: 15771

The "delta" there refers to a some subset of records, not to a subset of columns in all records. To add a new column you must reindex.

Are you aware of aliases so you can reindex into a new collection and when it's ready switch the alias to the new collection?

If you must keep the current data at all means then you could create a new collection with only id column and the new column (but then of course you have to deal with joins etc).

Upvotes: 2

Related Questions