CODEWITHSUNDEEP
solr
Baloo
Baloo

Reputation: 78

Query construction in custom Solr component plugin

I have developed a solr component which expands users query and adds additional clauses to the query. For this expansion we are making request to external REST api's. This query expansion logic is mainly in prepare() method. Everything works as expected in standalone mode. When we deploy this plugin in SolrCloud environment each shard is calling external REST api for query expansion.

My question is that can we make only one call to external REST api since its the same request sent from each shard to external service. How can we modify our component to make only one call per search request ?

Upvotes: 0

Views: 166

Answers (2)

sven.windisch
sven.windisch

Reputation: 442

In the prepare() method, right before your external API call, you can check RequestBuilder.isDistrib(). This boolean will be true for a request that is about to be distributed. You can then use this information to determine whether you can just execute the external request or you need to set one of the SolrCloud hosts that does this job.

How to determine the SolrCloud host to use for the external API? You could...

  • Hardwire one of the hosts into the component and check whether localhost is the hard wired host. That would unbalance the host load, though.
  • Have an arbitrary measurement that any component can check for itself, like hosts 1-10 fire the external request when the current minute is equal to the number of the host.
  • Even throw a dice in the search frontend, give the hostname to Solr via query parameter and have the component check this parameter (get it from ResponseBuilder.req.getParams()) against its local hostname.
  • You can get really creative there.

After you got an answer from the external API, you can use modifyRequest() to update all other hosts on the results.

Please read more in the Solr Wiki.

Upvotes: 1

Pavel Vasilev
Pavel Vasilev

Reputation: 1042

The approach you can use is like:

  • Rewrite your component into requestHandler (or just wrapper over standard requestHandler)

  • Make your requestHandler be aware of special flag which will trigger / not trigger your own custom logic. What I mean looks like this (I know it is not fancy):

    public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws Exception {
    ...
    SolrParams params = req.getParams();
    if (req.getParams().get("apiCallWasSent") == null) {
        makeApiCall(req, rsp);
        params = new ModifiableSolrParams(params);
        params.add("apiCallWasSent", "true");
        req.setParams(params);
    }
    ...
    super.handleRequestBody(req, rsp);
}
  • In my opinion those additional query clauses and everything what is related to the query itself should be handled by your own QParserPlugin. But component also can handle those clauses.

Upvotes: 0

Related Questions