CODEWITHSUNDEEP
luceneindexing
Jagadesh
Jagadesh

Reputation: 6807

Optimizing the number of Lucene index files

I am using Lucene to index the records from my database. I have a million records in my table called "Documents". The records will be accessed by particular users only. A real case scenario is that a single user can access a maximum of 100 records in the Documents table. Which of the following is a best practice for this scenario.

  1. Indexing all the 1 million records in Documents table as a single index file with the user information as one of the field in that index OR
  2. Creating user specfic indexes

Upvotes: 0

Views: 237

Answers (1)

milan
milan

Reputation: 12402

Sounds like you'll end up with a lot of indices in the second scenario, and if you want to search them concurrently, Lucene will have to keep a lot of files open, so you might easily hit your OS limit on the number of open files. If you decide to open/close them on demand, you might not benefit from caching and your search might be slow because of cold indices (or you prewarm them but again you might have a lot of overhead processing). I'd go with the first approach, Lucene can handle 1M documents in a single index.

Upvotes: 1

Related Questions