CODEWITHSUNDEEP
elasticsearchkibanaelasticsearch-aggregation
Tom
Tom

Reputation: 14250

Can Kibana visualizations aggregate by a meta field such as _size?

I'd like to visualize how our average elasticsearch document size is changing over time.

  1. We use ElasticSearch v7.1. We use the AWS ElasticSearch service and the Mapper Size Plugin is installed.
  2. I enabled the _size field on my index. ref
  3. In Kibana "Discover" I can find small documents using: _size: <900
    • Or I can query from the command line: curl -H 'Content-Type: application/json' -s http://es.example.com/logstash-2019.10.17/_search -d '{"query": {"range": {"_size": { "lt": 900 }}}}' | jq .

Now I want to create a Kibana visualization, for example a date histogram with the median _size, but Kibana "Visualize" won't let me select _size as the aggregation field. Is there a way to visualize the size of documents?

_size is a "meta-field".

_size The size of the _source field in bytes, provided by the mapper-size plugin.

Maybe "meta fields" aren't supported in Kibana?

I can aggregate by _size when not using Kibana "Visualize":

curl -H 'Content-Type: application/json' -s http://es.example.com/logstash-2019.10.17/_search -d '{
  "query": {
    "range": {
      "_size": {
        "gt": 10
      }
    }
  },
  "aggs": {
    "sizes": {
      "terms": {
        "field": "_size",
        "size": 10
      }
    }
  },
  "sort": [
    {
      "_size": {
        "order": "desc"
      }
    }
  ]
}' | jq .

Upvotes: 1

Views: 1845

Answers (2)

Sylwester O
Sylwester O

Reputation: 46

Question is old but looks not answered. The solution is:

  • go to Kibana Index Patterns
  • search and edit the one that you use in Kibana to retrieve data from Elasticsearch index (e.g. logstash-*)
  • edit --> Add field
    • use emit(doc['_size'].value) as a value
    • use Bytes as a format
  • save it

Now the _size value should be available as any other (_source) field at your Kibana visualizations/dashboards etc.

Upvotes: 3

Kamil Witkowski
Kamil Witkowski

Reputation: 2083

If you go in Kibana to Management->Elasticsearch index management you can see for specific index that:

enter image description here

Some fields does not have Aggretable on. Check other fields and see that they have circle in this column. And that's the reason why Kibana Visualize won't let you select _size as the aggregation field.

Maybe you could use https://www.elastic.co/guide/en/kibana/current/scripted-fields.html somehow to make it work by making additional field, if Kibana would allow you to use _size field inside of a script and that field would be Aggretable in a sense i mentioned earlier.

Upvotes: 2

Related Questions