CODEWITHSUNDEEP
solr
user3390906
user3390906

Reputation: 157

Search Solr for a large number of specific documens

I have read http://solr.pl/en/2011/12/19/do-i-have-to-look-for-maxbooleanclauses-when-using-filters/ and Too many boolean clauses exception in solr.

My Solr has about 2 million documents. I can retrieve specific documents by set query or filter query to the IDs of the specific documents. By doing this, I can find the facets and clusters among these specific documents. The query I set is:

id:1234567 or id:1234567 or id:2345678 ...

However, when I have, say 200 specific documents, Solr complains that my query has too many boolean clauses. Should I simply increase the maxBooleanClauses or there should be another approach for this kind of query?

Upvotes: 0

Views: 319

Answers (1)

Abhijit Bashetti
Abhijit Bashetti

Reputation: 8658

I had the same issue of too many boolean clauses exception in solr. You are having document ids, I had the acl ids(access control list)

Steps I have taken to overcome the issue is

  1. Increase the maxBooleanCLauses
  2. Used the in the filter query as &fq=acl:(id1 or id2...)

  3. If the count of acl is greater than 8000 then i am sending multiple request with the below logic

    StringBuilder queryString = new StringBuilder(5000); if (acls.length > 8000) { Integer aclCounter = 0; long requestCnt = Math.round(Math.ceil((acls.length / 8000.0))); for (int i = 0; i < requestCnt; i++) { queryString = new StringBuilder(acls.length * 10); queryString.append("&fq=acl:("); for (int j = 0; j < 8000 && aclCounter < acls.length; aclCounter++, j++) { queryString.append(acls[aclCounter]).append(APPEND_OR); } queryString.replace(queryString.length() - 4, queryString.length(), ")"); // build the solr request and sent it to get the response } }

Upvotes: 0

Related Questions