CODEWITHSUNDEEP
javalucenefull-text-search
VB_
VB_

Reputation: 45692

Lucene Out Of Memory

I'm using Lucene v4.10.4. I have pretty big index, it could be over few GBs. So I get OutOfMemoryError on initializing IndexSearcher:

try (Directory dir = FSDirectory.open(new File(indexPath))) { 

    //Out of Memory here!
    IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(indexDir));

How to tell Lucene's DirectoryReader to not load into memory more than 256 MB at once?

Log

Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.lucene.util.fst.BytesStore.<init>(BytesStore.java:68)
    at org.apache.lucene.util.fst.FST.<init>(FST.java:386)
    at org.apache.lucene.util.fst.FST.<init>(FST.java:321)
    at org.apache.lucene.codecs.blocktree.FieldReader.<init>(FieldReader.java:85)
    at org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:192)
    at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:441)
    at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:197)
    at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:254)
    at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:120)
    at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:108)
    at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:923)
    at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:53)
    at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:67)

Upvotes: 4

Views: 1190

Answers (1)

vahid
vahid

Reputation: 141

First you should check the current heap size of your JVM. 

java -XX:+PrintFlagsFinal -version | grep MaxHeapSize

If this number is not reasonable for your use case, you should increase it when running your program with -Xmx option of java command. A sample command to assign 8GB of heap memory would look like:

java -Xmx8g -jar your_jar_file

Hope this helps.

Upvotes: 1

Related Questions