Elastic Search Query with filter and aggregation returns no aggregations

Question

I'm using the node.js sdk to run an elasticsearch query that searches all fields for some text, then filters the results to a subset, then aggregates based on a custom index.

I've looked at the es documentation and several stack overflow questions that involve filters and aggregations to ensure my query syntax is correct, and I do not see a syntax error when the query is run. My query hits 2000+ documents in my dataset, but the response object contains no aggregations.

var search_text = "Chase Bank";
var doctype_selections = ["mortgage"]; 

es.search({
    index: index_list,
    type: 'page',
    body: {
        query: {
            filtered: {
                query: {
                    match: {
                        _all: {
                            "query": search_text,
                            "operator": "and"
                        }
                    },
                    filter: {
                        terms: {
                            document_type: doctype_selections
                        }
                    }
                }
            }
        },
        aggs: {
            "top_tag_hits":{
                terms: {
                    field: "agg_index",
                    size: agg_size
                },
                aggs: {
                    "hits":{
                        top_hits: {
                            size: agg_size
                        }
                    }
                }
            }
        },
    },
    explain: true,
    size: result_size, 
}).then(function (resp) {
    var hits = resp.hits.hits;
    var aggs = resp.aggregations;
    console.log(resp);
    response.send(aggs);
}, function (err) {
    console.trace(err.message);
    response.send(err.message);
});

From what I understand the correct order of operations in es would be query->filter query->aggregation on filtered query. If remove the filter and the aggregation the query by itself works as intended. If I remove the filter, but keep the aggregation and the query it works and returns aggregations. If I remove the aggregation and keep the filter, it also works as intended.

It seems as though the whole aggregation portion of the query is simply being ignored. What's the issue?

Answer 1

Here's how I was able to accomplish what I needed without using filter aggregations. I structure the query as a bool query and add seperate must and filter clauses:

es.search({
    index: index_list,
    type: 'page',
    body: {
        query: {
            bool: {
                must: [
                    {match: { _all: {
                            "query": search_text,
                            "operator": "and"
                            }}},
                ],
                filter: [ 
                    {terms: {document_type: doctype_selections}}
                ]
            }
        },
        aggs: {
            "top_tag_hits":{
                terms: {
                    field: "agg_index",
                    size: agg_size
                },
                aggs: {
                    "hits":{
                        top_hits: {
                            size: agg_size
                        }
                    }
                }
            }
        },
    },
    explain: true,
    size: result_size, 
}).then(function (resp) {
    var hits = resp.hits.hits;
    var aggs = resp.aggregations;
    console.log(resp);
    console.log(aggs);
    response.send(aggs);
}, function (err) {
    console.trace(err.message);
    response.send(err.message);
});

Answer 2

Aggregation works on search result which are returned after query. You can look to post_filter its possible to get results from query and then calculate aggregations and then filter

Also each aggregation can have its own filter

Answer 3

Maybe you need to use a filter aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-filter-aggregation.html