CODEWITHSUNDEEP
lucene.net
roryf
roryf

Reputation: 30150

How does Lucene.NET IndexSearcher read and store index in memory?

I am using a read-only Lucene.NET index in a web application and creating a single IndexSearcher for all requests. The creation process for this is thread-safe, as are the IndexSearcher methods according to the docs.

My question is around how the IndexSearcher stores the index in memory after opening. Does it read the entire index into memory on creation?

I ask because the application uses (potentially) hundreds of different indices, at approx. 27MB (on disk) each, with a new IndexSearcher for every one. I want to be aware of how this will affect the memory usage of the worker process.

Upvotes: 1

Views: 735

Answers (1)

Jf Beaulac
Jf Beaulac

Reputation: 5246

The IndexSearcher will not store the entire index in memory, but it's underlying reader caches some information about Fields, Terms and Segments. The amount of memory required will vary a lot depending of what's in your index, etc.

Using a really high amount of indices like that will affect total memory consumed by the process. It is generally better to have fewer bigger indices.

If you intend to use several hundred indices, I would recommend to test the scalability of your approach with real production data first.

Upvotes: 2

Related Questions