CODEWITHSUNDEEP
solrsolrcloudsolr5
Qedrix
Qedrix

Reputation: 473

Solr Suggest Component and OutOfMemory Error

I am using the Solr Suggester component in Solr 5.5 with a lot of address data. My Machine has allotted 20Gb RAM for solr and the machine has 32GB RAM in total.

I have an address book core with the following vitals -

"numDocs"=153242074
"segmentCount"=34
"size"=30.29 GB

My solrconfig.xml looks something like this -

<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
  <str name="name">mySuggester1</str>
  <str name="lookupImpl">FuzzyLookupFactory</str>
  <str name="storeDir">suggester_fuzzy_dir</str>

  <!-- Substitute these for the two above for another "flavor"
    <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
    <str name=?indexPath?>suggester_infix_dir</str>
  -->

  <str name="dictionaryImpl">DocumentDictionaryFactory</str>
  <str name="field">site_address</str>
  <str name="suggestAnalyzerFieldType">suggestType</str>
  <str name="payloadField">property_metadata</str>
  <str name="buildOnStartup">false</str>
  <str name="buildOnCommit">false</str>
</lst>
<lst name="suggester">
  <str name="name">mySuggester2</str>
  <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
  <str name="indexPath">suggester_infix_dir</str>

  <str name="dictionaryImpl">DocumentDictionaryFactory</str>
  <str name="field">site_address_other</str>
  <str name="suggestAnalyzerFieldType">suggestType</str>
  <str name="payloadField">property_metadata</str>
  <str name="buildOnStartup">false</str>
  <str name="buildOnCommit">false</str>
</lst>
</searchComponent>

The handler is defined like so -

<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
<lst name="defaults">
  <str name="suggest">true</str>
  <str name="suggest.count">10</str>
  <str name="suggest.dictionary">mySuggester1</str>
  <str name="suggest.dictionary">mySuggester2</str>
  <str name="suggest.collate">false</str>
  <str name="echoParams">explicit</str>
</lst>
<arr name="components">
  <str>suggest</str>
</arr>
</requestHandler>

Problem Statement

Every time I try to build the suggest index using the suggest.build=true url parameter, I end up with an OutOfMemory error. I have no clue how I can make this work with the current setup. Can anyone explain why this is happening? And how can I fix this issue?

Upvotes: 1

Views: 218

Answers (1)

elyograg
elyograg

Reputation: 789

On the mailing list, the critical info is that the OOME is due to java heap space.

This means that you either need to increase your heap size or reduce the amount of heap that Solr needs. It is not always possible to reduce the heap requirements, but here are some ideas:

https://wiki.apache.org/solr/SolrPerformanceProblems#Reducing_heap_requirements

Upvotes: 0

Related Questions