CODEWITHSUNDEEP
solrstorage
dotancohen
dotancohen

Reputation: 31511

Solr long term storage, or export function

Our Solr application adds about 1 GiB of data to the Solr index daily. The application only accesses the last N days' data, however all the data must be saved for future features. To save space on the main server (expensive) I would like to export the data and save it on local storage (cheap). MySQL has the wonderful mysqldump feature which makes the data easy to export and import as per need. Does Solr have a similar feature that can be activated without shutting down the Solr application?

The specific critical features that I am interested in:

  1. Works while Solr is running, no downtime.

  2. Export subset of data, such as datetime:[1356220800 TO 1356393600]

  3. Import the data

The specific non-critical features that I am interested in:

  1. Ability to search the exported data to find a subset to insert (using either common GNU tools such as grep or a small standalone tool.

  2. Intrinsic data compression.

Upvotes: 0

Views: 289

Answers (1)

Jayendra
Jayendra

Reputation: 52799

You can check upon :-

  • Solr backup - Will allow you the backup the index on the same server. You can reuse the index as it is a full fledged Solr Index
  • Replication - You can replicate every periodically and maintain the index on a different Solr server. This is basically export and import both as it needs Solr running and would be hosted for you to use.
  • You can always use the Delete query to delete (all records before the date) the data from Solr.

In Addition

  • There is no time specific export or dump functionality. However, you can perform the above daily, weekly and you would have only the data within that period.
  • All the above would work on Solr without any downtime.
  • The index can be query with other Solr instance or Luke.

Upvotes: 3

Related Questions