CODEWITHSUNDEEP
lucenesolr
sesmic
sesmic

Reputation: 928

Solr commit taking too long

My commit seems to be taking too much time, if you notice from the Dataimport status given below to commit 1000 docs its taking longer than 24 minutes

<str name="status">busy</str>
<str name="importResponse">A command is still running...</str>
<lst name="statusMessages">
<str name="Time Elapsed">0:24:43.156</str>
<str name="Total Requests made to DataSource">1001</str>
<str name="Total Rows Fetched">1658</str>
<str name="Total Documents Skipped">0</str>
<str name="Full Dump Started">2011-06-07 09:15:17</str>
<str name="">
Indexing completed. Added/Updated: 1000 documents. Deleted 0 documents.
</str>
</lst>

What can be causing this, I have tried looking for a reason or a way to improve this, but am just not able to find. At this rate my documents would never get indexed, given that I have more than 100,000 records coming into the database every hour.

Regards, Rohit

Upvotes: 1

Views: 2068

Answers (2)

sesmic
sesmic

Reputation: 928

Have added optimize=false in the data import handler. This has made things faster to some extent, now I am optimizing only during off peak hours.

Upvotes: 0

Bob Yoplait
Bob Yoplait

Reputation: 2499

I don't know if you use solrj

public abstract class SolrServer

but if you do, you really need to index by chuncks/collections:

  public UpdateResponse add(Collection<SolrInputDocument> docs )

and not one by one

Upvotes: 1

Related Questions