spring-data-elasticsearch using Criteria with nested Objects

Question

I'm quite new to elasticsearch and spring-data-elasticsearch and I have some problems querying for nested objects.

I use the ElasticSearch repository to save an nested model instance in elasticsearch. As a result there is only one entry in elasticsearch containing all data, as far I understood this means I have a nested document.

I need to implement a relatively complex query using Criteria to build up the query iteratively. When I try to access a nested property using the dot notation like

startPoint.providingTimeRange.startTime

I don't get any search results (but matching data is present in elasticsearch).

In Spring Data Elastic Search with Nested Fields and mapping I found the capability of spring-data-elasticsearch to query for nested objects using the nestedQuery.

Is there any way to combine the Criteria search with the nestedQuery?

Thank you in advance, Christoph

Update 1: To provide some example code I created a demo project which contains similar nested objects like in my real project: https://github.com/empulse-gmbh/elasticsearchtest

The example is about to find FoodTrucks which are in a certain place in a certain time.

In this example I used a Repository to persist my nested entities.

I used two (both not working) approaches to query for nested objects:

        /*
         * add search criteria by timerange. it is assumed in timerange from
         * is always before to.
         */
        TimeRange searchTimeRange = foodTruckSearch.getSearchTimeRange();
        if (searchTimeRange != null) {

            String startTimePath = "locationPoint.timeRange.from";
            String endTimePath = "locationPoint.timeRange.to";

            searchCriteria = searchCriteria.and(
                    new Criteria(startTimePath).between(searchTimeRange
                            .getFrom().getTime(), searchTimeRange.getTo()
                            .getTime())).or(
                    new Criteria(endTimePath).between(searchTimeRange
                            .getFrom().getTime(), searchTimeRange.getTo()
                            .getTime()));
        }

and:

    TimeRange searchTimeRange = foodTruckSearch.getSearchTimeRange();
    if (searchTimeRange != null) {

        String startTimePath = "locationPoint.timeRange.from";
        String endTimePath = "locationPoint.timeRange.to";

        searchQuery.must(nestedQuery(
                "locationPoint",
                boolQuery().should(
                        rangeQuery(startTimePath).from(
                                searchTimeRange.getFrom().getTime()).to(
                                searchTimeRange.getTo().getTime())).should(
                        rangeQuery(endTimePath).from(
                                searchTimeRange.getFrom()).to(
                                searchTimeRange.getTo()))));

    }

Update2: Thanks to Mohsin Husen I was able to use the nested search. The search by timerange works now. Additionally to Mohsin's suggestions I had to change the way of creating and starting the local Elasticsearch instance:

elasticSearchNode = NodeBuilder.nodeBuilder().clusterName("foodtruck-test").local(true).build();

Without giving the clusterName nested documents are not created.

My next (hopefully last) problem is using the geo_distance search filter. If I got it right in Elasticsearch I use queries for fuzzy searches like 'like' or 'range' and filters for yes/no or distance filters.

So I tried to use QueryBuilder and FilterBuilder together with spring-data-elasticsearch and failed.

According to the examples in the documentation I refactored my models so the Location class uses the spring-data-elasticsearch object GeoPoint.

For searching I use this:

    /*
     * add search criteria for radius search
     */
    FilterBuilder searchFilter = null;
    if (foodTruckSearch.getLatitude() != null
            && foodTruckSearch.getLongitude() != null) {

        if (foodTruckSearch.getSearchRadiusInKilometers() == null) {
            foodTruckSearch.setSearchRadiusInKilometers(5);
        }

        searchFilter = geoDistanceFilter("location.point")
                .distance(
                        foodTruckSearch.getSearchRadiusInKilometers()
                                + "km").lat(foodTruckSearch.getLatitude())
                .lon(foodTruckSearch.getLongitude());

    }

    if (searchFilter != null) {
        NativeSearchQuery nativeSearchQuery = new NativeSearchQuery(
                searchQuery, searchFilter);
        return IteratorUtils.toList(foodTruckRepository.search(
                nativeSearchQuery).iterator());
    } else {
        return IteratorUtils.toList(foodTruckRepository.search(searchQuery)
                .iterator());
    }

The exception I get is:

org.elasticsearch.action.search.SearchPhaseExecutionException: Failedtoexecutephase[query_fetch],
allshardsfailed;shardFailures{
    [1][searchexample][0]: SearchParseException[[searchexample][0]: query[MatchNoDocsQuery],
    from[0],
    size[10]: ParseFailure[Failedtoparsesource[{
    "from": 0,
    "size": 10,
    "query": {
        "bool": {

        }
    },
    "post_filter": {
        "geo_distance": {
            "location.point": [6.9599115,
            50.9406645],
            "distance": "10km"
        }
    }
}]]];nested: QueryParsingException[[searchexample]failedtofindgeo_pointfield[location.point]]   ;
}

Is there any example available how to use geo_distance (nested?) filters with spring-data-elasticsearch?

Answer 1

The exception was caused by giving a default mapper in

return new ElasticsearchTemplate(elasticSearchNode.client());

Removing this wrong mapper caused spring-data-elasticsearch to put the correct mapping into Elasticsearch.

If you want to have an entity enabled for geo_distance search use the

@GeoPointField

annotation to mark your field containing the lat/lon values.

Additionally the FoodTruckServiceImpl was changed to use filters, not queries.

The project in GitHub is updated and is a fully functional example. Feel free to use the FoodTruck example for first steps with spring-data-elasticsearch.

Answer 2

Without reading in details, i caught some of the problems in your code as below

1) As you are using multiple level nested document you have to use snapshot version (which you are using) as spring data elasticsearch M1 release has bug which is fixed. (https://jira.springsource.org/browse/DATAES-53)

2) You are declaring nested type wrong in your entities, it has to be as follow

    @Field(type=FieldType.Nested)
    private LocationPoint locationPoint;

    @Field(type = FieldType.Nested)
    private TimeRange timeRange;

3) Criteria query wont deal with nested object, it is built for dealing with simple entities as it is using query_string for querying data. hence use elasticsearchTemplate or repository to query nested objects.

4) You have to specify full path for nested object

e.g, while searching for locationPoint.timeRange.from and locationPoint.timeRange.to you have to use as follows

 String startTimePath = "locationPoint.timeRange.from";
    String endTimePath = "locationPoint.timeRange.to";

    searchQuery.must(nestedQuery(
            "locationPoint.timeRange",
            boolQuery().should(
                    rangeQuery(startTimePath).from(
                            searchTimeRange.getFrom().getTime()).to(
                            searchTimeRange.getTo().getTime())).should(
                    rangeQuery(endTimePath).from(
                            searchTimeRange.getFrom()).to(
                            searchTimeRange.getTo()))));

Hope this will solve your issue.