CODEWITHSUNDEEP
elasticsearch
OneMoreError
OneMoreError

Reputation: 7728

Elastic Search Limiting the records that are aggregated

I am running an elastic search query with aggregation, which I intend to limit to say 100 records. The problem is that even when I apply the "size" filter, there is no effect on the aggregation.

GET /index_name/index_type/_search
{
  "size":0,
  "query":{
    "match_all": {}
  },
  "aggregations":{
    "courier_code" : {
      "terms" : {
        "field" : "city"
      }
  }
}}

The result set is 

{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 10,
    "successful": 10,
    "failed": 0
  },
  "hits": {
    "total": 10867,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "city": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "Mumbai",
          "doc_count": 2706
        },
        {
          "key": "London",
          "doc_count": 2700
        },
        {
          "key": "Patna",
          "doc_count": 1800
        },
        {
          "key": "New York",
          "doc_count": 1800
        },
        {
          "key": "Melbourne",
          "doc_count": 900
        }
      ]
    }
  }
}

As you can see there is no effect on limiting the records on which the aggregation is to be performed. Is there a filter for, say top 100 records in Elastic Search.

Upvotes: 1

Views: 325

Answers (1)

imotov
imotov

Reputation: 30163

Search operations in elasticsearch are performed in two phases query and fetch. During the first phase elasticsearch obtains results from all shards sorts them and determines which records should be returned. These records are then retrieved during the second phase. The size parameter controls the number of records that are returned to you in the response. Aggregations are executed during the first phase before elasticsearch actually knows which records will need to be retrieved and they are always executed on all records in the search. So, it's not possible to limit it by the total number of results. If you want to limit the scope of aggregation execution you need to limit the search query instead changing retrieval parameter. For example, if you add a filter to your search query that will only include records from the last year, aggregations will be executed on this filter.

It's also possible to limit the number of records that are analyzed on each shard using terminate_after parameter, however you will have no control on which records will be included and which records wouldn't be included into results, so this option is most likely not what you want.

Upvotes: 1

Related Questions