CODEWITHSUNDEEP
solrsolrjsolrnetsolrcloud
Javadroider
Javadroider

Reputation: 2450

Solr: To get all records

I am trying to upgrade my Solr 4.x version to 5.2.1 Solrcloud implementation. I had written following code to get all the results from Sorl query which works well in Solr single instance mode.

SolrQuery query = new SolrQuery();
query.setQuery("*:*");
query.addSort("agent_status", ORDER.desc);
query.addFilterQuery("account_id:\"" + accountId + "\"");
query.set("rows", Integer.MAX_VALUE);

But code will not work well in SolrCloud implemenation.It throws following exception.

2015-08-14 16:44:45,648 ERROR [solr.core.SolrCore] - [http-8080-8] : java.lang.NegativeArraySizeException
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:58)
at org.apache.lucene.util.PriorityQueue.<init>(PriorityQueue.java:39)
at org.apache.solr.handler.component.ShardFieldSortedHitQueue.<init>(ShardDoc.java:113)
at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:972)
at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:750)
at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:729)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)

I found that it is failing because of query.set("rows", Integer.MAX_VALUE) statement.People suggested me to use pagination. But, I can not afford doing pagination as there will be too many changes at UI side.

There is one more way where I can first query with some small number & get total number of documents using response.getResults().getNumFound() method & try setting that value to setRows method.But this approach will increase one more call to server.

Is there any other way I can solve this problem?

Upvotes: 1

Views: 1556

Answers (1)

cahilltr
cahilltr

Reputation: 128

You can always set your rows to be a large value that would encompass your results. Integer.MAX_VALUE will not work due to the size limits of Java Arrays (see here) and the Lucene Priority Queue (see lines 42 - 58).

Solr-534 requested to have essentially what your asking for; there is some good conversation about why and why-not such a feature would be good.

A better question might be how many documents can the UI hold without becoming unusable? However many documents that is, would be a good value for your query to return.

Upvotes: 1

Related Questions