CODEWITHSUNDEEP
elasticsearchelasticsearch-aggregation
Evaldas Buinauskas
Evaldas Buinauskas

Reputation: 14077

Speed up Elasticsearch terms aggregation / SELECT DISTINCT

I'd like to know if it's possible to speed up Elasticsearch terms aggregation.

My actual goal is to select multiple distinct fields for some query, for instance, that's a query I'd be using something like this. It potentially could include nested documents later:

{
  "query" : {
    "match" : {
      "Company" : "samsung"
    }
  },
  "aggs" : {
    "Products" : {
      "terms" : {
        "field" : "ProductCode"
      }
    },
    "Countries" : {
      "terms" : {
        "field" : "CountryCode"
      }
    }
  }
}

So I'd find all relevant documents that have samsung as its company and aggregate all of its productCodes and countryCodes (They are integers).

Is there a way to speed such a query? I don't care about the actual doc_count brought back, all I need is distinct values. Perhaps there's some sort of hint or a better aggregation to get this done?

Upvotes: 0

Views: 1388

Answers (1)

Chris Heald
Chris Heald

Reputation: 62658

If you use a filter query rather than a normal query, it'll improve your initial runtime:

{
  "query" : {
    "filtered": {
      "term": {"yourField": "samsung"}
    }
  },
  "aggs" : {
    // ...
  }
}

This is because filter queries are pass/fail, rather than having to be scored.

The other things you can do are set your mapping to store these fields as doc_values, which will substantially reduce the memory requirements necessary to perform those aggregations, and to set up eager loading on those fields, so that ES will preload that data and have it available for aggregation rather than having to load it on demand, which will improve response times.

We haven't experienced much in the way of downsides with doc_values - they are a clear across-the-board win when you're doing any significant aggregation work. Eager loading will result in slower refreshes, but may be acceptable if you're more sensitive to timely queries than to refresh speed.

Upvotes: 2

Related Questions