Elasticsearch: pagination on unique values

Question

I am building some Logging and monitoring product for my employer and using ES as backend. now finding unique value of each/any attribute is core part of business logic I have in hand.

let's say I want unique dst_ip, to achieve that,

• I have used "index":"not_analyzed" for selected fields

• Api used to get unique count

  http:// 127.0.0.1:9200/es-server/Events/_search -d '{
    "aggs": {
      "dst_ip_count": {
        "cardinality": {
          "field": "dst_ip"
        }
      }
    },
    "size": 0
  }'
  

• Api used to fetch those values

  http:// 127.0.0.1:9200/es-server/Events/_search -d '{
    "fields": [
      "dst_ip"
    ],
    "facets": {
      "terms": {
        "terms": {
          "field": "dst_ip",
          "size": 1116,
          "order": "count"
        }
      }
    },
    "size": 1116
  }'
  

here 1116 is received from first API. now here the count is very small but in production environment this count goes greater than 2 thousand . which results in slow query response.

do we have any other way to fetch such values with pagination in built like we have in search query with size and from?

Answer 1

Elasticsearch does not support pagination for aggregation results, only for the documents themselves. If you want to return all aggregation results, set "size": 0.