CODEWITHSUNDEEP
elasticsearch
Fizer Khan
Fizer Khan

Reputation: 92875

How do I get histogram with count in ElasticSearch

We are trying to generate histogram in ElasticSearch using Histogram aggregation

We are monitoring our API performance with ElasticSearch. A sample set of documents that we store are

{"name": "GET /login", "avg":2.2, "count": 5}
{"name": "GET /login", "avg":1.5, "count": 3}
{"name": "GET /login", "avg":6.9, "count": 1}
{"name": "GET /login", "avg":3.1, "count": 1}

Where

Histogram Issue

When I run histogram for GET /login alone, with interval of 1 second, I get

 1-2 seconds - 1
 2-3 seconds - 1
 3-4 seconds - 1
 5-6 seconds - 1

However, what we are doing is, we are only storing the average with the count (to save space and not duplicate the records). But in reality, what we would like to get is

 1-2 seconds - 3
 2-3 seconds - 5
 3-4 seconds - 1
 5-6 seconds - 1

Mapping for our current hist index

PUT hist/_mapping/t
{
  "properties": {
    "name": {
      "type": "string"
    },
    "avg": {
      "type": "long"
    },
    "count": {
      "type": "long"
    }
  }
}

We uses following query to get histogram.

GET /hist/t/_search
{
  "aggs": {
    "avgs": {
      "histogram": {
        "field": "avg",
        "interval": 1
      }
    }
  }
}

Percentile Issue

The same problem appears for percentile as well.

1.5, 2.2, 3.1, 6.9 - The 75th percentile is reported as 3.1 second. 1.5, 1.5, 1.5, 2.2, 2.2, 2.2, 2.2, 2.2, 3.1, 6.9 - The real 75th percentileis 2.2 seconds

We uses following query to get percentile.

GET /hist/t/_search
{
  "aggs": {
    "avgs": {
      "percentiles": {
        "field": "avg"
      }
    }
  }
}

How do I do it in elastic search?

Upvotes: 2

Views: 2034

Answers (1)

Val
Val

Reputation: 217424

What you're getting now is the document count for each histogram bucket. You can add one sum sub-aggregation on the count field and you should get what you expect. Try this:

POST /hist/t/_search
{
  "aggs": {
    "avgs": {
      "histogram": {
        "field": "avg",
        "interval": 1
      },
      "aggs": {
        "sum_of_count": {
          "sum": {
            "field": "count"
          }
        },
        "perc_of_count": {
          "percentiles": {
            "field": "count"
          }
        }
      }
    }
  }
}

Upvotes: 3

Related Questions