CODEWITHSUNDEEP
elasticsearchelasticsearch-aggregation
Amal Kumar S
Amal Kumar S

Reputation: 16065

ElasticSearch - How to display an additional field name in aggregation query

How can I add a new key called 'agency_name' in my output bucket.

I am running an aggregation code as shown below

{
  "aggs": {
    "name": {
      "terms": {
        "field": "agency_code"
      }
    }
  }
}

I will be getting the out put as

"aggregations": {
    "name": {
        "doc_count_error_upper_bound": 130,
        "sum_other_doc_count": 39921,
        "buckets": [
            {
                "key": "1000",
                "doc_count": 105163
            },
            {
                "key": "2100",
                "doc_count": 43006
            }
        ]
    }
}

While displaying I need to show the agency name, code and doc_count

How can I modify the aggregation query so that I could get the below format. I am new to ElasticSearch, not sure how to fix this 

"aggregations": {
    "name": {
        "doc_count_error_upper_bound": 130,
        "sum_other_doc_count": 39921,
        "buckets": [
            {
                "key": "1000",
                "doc_count": 105163,
                "agency_name": 'Agent 1'
            },
            {
                "key": "2100",
                "doc_count": 43006,
                "agency_name": 'Agent 2'
            }
        ]
    }
}

Sample Data in ElasticSearch (fields are analysed)

{

    "_index": "feeds",
    "_type": "news",
    "_id": "22005",
    "_version": 1,
    "_score": 1,
    "_source": {
        "id": 22005,
        "name": "Test News",
        "agency_name": "Agent 1",
        "agency_code": "1000",
    }

}

Upvotes: 29

Views: 15822

Answers (6)

Hari Ram
Hari Ram

Reputation: 305

This might be a bit old, but from ES 7.12 onwards, we have multi_terms aggregation. Your query should look something like this.

{
  "aggs": {
    "group_by_agency": {
      "multi_terms": {
      "terms": [
        {"field": "agency_code"},
        {"field": "agency_name"}
      ]
    }
  }
}

Upvotes: 0

Chukwuma Nwaugha
Chukwuma Nwaugha

Reputation: 715

What I do is use something like the following query: 

"aggs" : {
    "products" : {
      "filter" : { "term": { "item.category": "children" }},
      "aggs" : {
        "count" : {
          "terms" : {
            "script": "doc['item.id'].value + ':' + doc['item.name'].value"
          }
        }
      }
    }
  }

Which returns something like this:

...
"aggregations" : {
    "products" : {
      "doc_count" : 1050,
      "count" : {
        "doc_count_error_upper_bound" : 0,
        "sum_other_doc_count" : 0,
        "buckets" : [
          {
            "key" : "x2_90QBj9k:Baby Oil",
            "doc_count" : 45
          },
          ...
        ]
...

And then I can use a string operation on bucket[i]["key"], for each i in a loop, to extract the relevant field.

Upvotes: 0

Vishwa
Vishwa

Reputation: 31

This is old post, however, I ran into same issue and I followed what is given at https://www.elastic.co/guide/en/elasticsearch/reference/current/agg-metadata.html. Add metadata details and it is return as part of the result above bucket. Hope it will help someone in future.

Upvotes: 1

Rajas Agashe
Rajas Agashe

Reputation: 371

You can use the top hits aggregation like in the link below. The format will be slightly different since creating the extra aggregation will embed the agency name under another 'hits' key.

Adding additional fields to ElasticSearch terms aggregation

{
  "aggs": {
    "name": {
      "terms": {
        "field": "agency_code"
      },
      "aggs": {
        "agency_names" : {
           "top_hits": {
                size: 1, 
                _source: {
                    include: ['agency_name']
                }
            }
         } 
       }
    }
  }
}

Upvotes: 20

Andrew White
Andrew White

Reputation: 53516

ES has no way of knowing agency_name and agency_code map one-to-one. Therefore I would recommend a number of possible strategies.

  • Don't analyze agency_name and use the term agg over that field. I would be surprised if you actually need to do tokenization of the agency_name.
  • Store the id to name mapping in a relational database or a flat file cache and do the join client side
  • Store the agency documents as another type and make two calls. The first to get the ids and then a second to lookup the agencies by id

As Aditya Patel mentioned above, parent child relationships may help out as well but I believe you will still have to use one of the above strategies to resolve the id->name mapping.

Upvotes: 1

Aditya Patel
Aditya Patel

Reputation: 569

I think you would need to add another "aggs" to it. But it would not be in the format in which you want but as another field in the output , reason being currently you are aggregating based on "agency_code" and the doc_count shows how many times the particular agency code occurs. Now when you want to aggregate it based on "agency_name" the field might in different documents than "agency_code" and in different numbers as well , if they always exist in pair than this parent-child indexing might be of some help.

https://www.elastic.co/guide/en/elasticsearch/guide/current/indexing-parent-child.html

Upvotes: 2

Related Questions