CODEWITHSUNDEEP
solr
joaguilar
joaguilar

Reputation: 31

Override "fl" parameter in Solr using SolrParams in a custom SearchComponent

I have an interesting use case for a Solr implementation we have, where there are some fields in the Solr Schema that shouldn't be returned when doing a query. The ideal solution is to change the calling program so it doesn't query for &fl=score like it does now, and only requests the necessary fields, but that won't happen in the short term so in the meantime we have to filter out some fields from the Solr response.

The approach we think has the smallest performance impact (let me know if there is a better way to do this), is to override the &fl= parameter so it lists all the fields but the ones that should be filtered out. For this, we added a new SearchComponent to the RequestHandler components list that modifies the &fl parameter. The issue we ran into with this approach is that once we get the SolrParams from the SolrQueryRequest, it cannot be modified (which is I think is the right thing to do, since it could be changing something another SearchComponent relies on). But we still need to find a way to remove these extra fields.

So, this is the code we started to write:

public void prepare(ResponseBuilder rb) throws IOException {
        SolrQueryRequest req = rb.req;
        SolrParams params = req.getParams();
        String fl = params.get("fl");
        //Remove the "fl" parameter from params and replace it with a new list:
        //Cannot be done"
        ...

And ran into the issue of not being able to add to the SolrParams.

As a plan B, that same SearchComponent is removing the fields in the process() method, but doing it this way is slower. The code has to go through the resulting SolrDocumentList, and for each SolrDocument call removeFields(), something similar to: (simplified code)

public void process(ResponseBuilder rb) throws IOException {
    ...
    SolrQueryResponse rsp = rb.rsp;
    NamedList values = rsp.getValues();
    SolrDocumentList docs = (SolrDocumentList) values.get("response");
    Iterator<SolrDocument> docsIterator = sdoclist.iterator();
    while (docsIterator.hasNext()) {
        SolrDocument sd = sdocIterator.next();
        sd.removeFields(field);
        ...

Any ideas on how/if this can be achieved?

Thanks for any suggestion!

Upvotes: 3

Views: 2707

Answers (2)

Ferran Gal&#237;
Ferran Gal&#237;

Reputation: 21

Try removing the fields that you don't want from the ReturnFields object. For example, something like this:

@Override
public void process(ResponseBuilder rb) throws IOException {
    String fl = rb.req.getParams().get(CommonParams.FL);
    List<String> fields = Lists.newArrayList(fl.split(","));
    List<String> newFields = Lists.newArrayList();
    for (String field : fields) {
        if (!field.equals("score")) {
            newFields.add(field);
        }
    }
    String newFl = Joiner.on(",").join(newFields);
    ReturnFields returnFields = new ReturnFields(newFl, rb.req);
    rb.rsp.setReturnFields(returnFields);
}

I've set the custom SearchComponent in "last-components" at solrconfig.xml.

P.S: I was using guava libraries for Lists and Joiner.

Upvotes: 0

Fermin Silva
Fermin Silva

Reputation: 3377

With your own SearchHandler you can specify invariants (things that will always be fixed no matter the request) on any query parameter, among which there is the &fl.

It's something in the lines of:

<requestHandler name="filtered" class="solr.StandardRequestHandler">
    <lst name="invariants">
      <str name="fl">score,id,something_else,etc.</bool>
    </lst>

  </requestHandler>

More documentation: http://wiki.apache.org/solr/SearchHandler

The only problem is that, for now, there's no negative fl parameter (i.e. return all fields except those i'm telling you). https://issues.apache.org/jira/browse/SOLR-3191

Finally, to specify which SearchHandler you want to use at query time, simply add &qt=filtered (or the name you used for it)

Upvotes: 1

Related Questions