Elasticsearch: Filter on multiple arrays of nested objects

Question

I'm new to elasticsearch and am having trouble with making a query I need. I've read through querying, filtering, booling querying, and nested objects but I'm still a bit stumped. I'm using the php composer package for elasticsearch, but I think it's more of an issue with my set up.

I'm trying to filter objects based on a property they have and properties that their nested objects have.

an example object I would have is as follows:

{
    'id' : '1',
    'title' : 'real catchy title',
    'description' : 'description goes here',
    'content' : [
        'id' : '1',
        'title' : 'foo',
        'subtitle' : 'bar',
        'text' : 'legit full text',
        'sidebar' : 'whatever yo!',
    ],
    'pages' : '12',
    'departments' : [
        {
            'id' : '1',
            'name' : 'foo',
            'description' : 'lorem ipsum'
        },
        {
            'id' : '2',
            'name' : 'bar',
            'description' : 'lorem ipsum'
        }
     ]
    'themes' : [ 
        {
            'id' : '1',
            'name' : 'foo',
            'description' : 'lorem ipsum',
            'month' : '3'
        },
        {
            'id' : '2',
            'name' : 'bar',
            'description' : 'lorem ipsum',
            'month' : '2'
        }
   ]
}

and I'm trying to filter on it based on if the page matches a given number, and if it has a department with a given department id and has a theme with a given theme id.

In order to do this, I have my mapping as:

[
    'index' : ,'reusable_content'
    'body' => [
        'mappings' => [
            'reusable_content' => [
                '_all' => ['analyzer' => 'english'],
                'properties' => [
                    'departments' => [
                        'type' => 'nested', 
                        "include_in_parent" => true
                    ],
                    'themes' => [
                        'type' => 'nested', 
                        "include_in_parent" => true
                    ]
                ]
            ]
        ]
    ]
]

I insert it and then, I've tried a few ways to query it after I add a few entries.

I've tried using boolean nested filters:

[
            'index' => $this->getIndexName(),
            'type' => 'reusable_content',
            'body' => [
                'query' => [
                    'filtered' => [
                        'query' => ['match_all' => []],
                        'filter' => [
                            'bool' => [
                                'should' => [
                                    'nested' => [
                                        'path' => 'themes',
                                        'filter' => [
                                            'bool' => [
                                                'must' => [
                                                    'term' => [
                                                        'id' => $themeId
                                                    ]
                                                ]
                                            ]
                                        ]
                                    ],
                                    'nested' => [
                                        'path' => 'departments',
                                        'filter' => [
                                            'bool' => [
                                                'must' => [
                                                    'term' => [
                                                        'id' => $departmentId
                                                    ]
                                                ]
                                            ]
                                        ]
                                    ]
                                ]
                            ]
                        ]
                    ]
                ]
            ]
        ]

this doesn't return any results.

I've tried using simple methods like a bool must query for the filter as well:

[
            'index' => $this->getIndexName(),
            'type' => 'reusable_content',
            'body' => [
                'query' => [
                    'filtered' => [
                        'query' => ['match_all' => []],
                        'filter' => [
                            'bool' => [
                                'must' => ['term' => ['pages' => $pages]],
                                'must' => ['term' => ['departments.id' => $departmentId]],
                                'must' => ['term' => ['themes.id' => $themeId]],
                            ]
                        ]
                    ]
                ]
            ]
        ]

this mostly works, however it ignores the pages filtering, however if I only have the 'must' => ['term' => ['pages' => $pages]], for the filter and ignore the id fields, the pages filter does work.

I'm pretty new to elastic search so if I'm making some weird assumption or doing something very wrong please let me know so I can learn, and also if you need more information please just ask!

Answer 1

Your first query seems fine. Problem seems with your analyzer and terms query. Terms filter looks for exact match in the inverted index, i.e. suppose theme name was 'happiness' then english analyzer may index it as just 'happy'. So terms filter would need the exact token that is indexed which in the above case is 'happy' and not 'happiness'. I would recommend to change terms filter to a match query because it analyzes the string first, and if it returns the result consider changing the analyzer or keep using match query.