Spring-data-elasticsearch Metadata _score

Question

When querying Elasticsearch from Spring-Data I would like to get the _score

If we take the simple following Class:

@Document(indexName = "test", type = "el_test")
public static class ElTest{
    private long id;
    private String myField;
}

With a JUnit Test

@Test
public void testScore() throws Exception {
    elasticsearchTemplate.index(new IndexQueryBuilder()
            .withIndexName("test")
            .withObject(new ElTest("first value"))
            .build());
    elasticsearchTemplate.index(new IndexQueryBuilder()
            .withIndexName("test")
            .withObject(new ElTest("second value"))
            .build());
    elasticsearchTemplate.index(new IndexQueryBuilder()
            .withIndexName("test")
            .withObject(new ElTest("third"))
            .build());

    SearchQuery query = new NativeSearchQueryBuilder()
            .withQuery(QueryBuilders.matchQuery("myField","second value"))
            .build();

    List<ElTest> els = elasticsearchTemplate.queryForList(query, ElTest.class);
    assertEquals(2, els.size());
}

This will create 3 entries in Elasticsearch. The query will retrieve two values with different scores.

If we put the request directly in Elasticsearch:

POST /test/_search
{
  "query": { 
    "match" : {
      "myField" : {
        "query" : "second value",
         "type" : "boolean"
      }
    }
  }
}

And the response from Elasticsearch:

{
  "took": 15,
  "timed_out": false,
  "_shards": {
     "total": 5,
     "successful": 5,
     "failed": 0
   },
"hits": {
  "total": 2,
  "max_score": 0.8838835,
  "hits": [
     {
        "_index": "test",
        "_type": "el_test",
        "_id": "AVKmmYCL3xnXT_BGRA3T",
        "_score": 0.8838835,
        "_source": {
           "id": 0,
           "myField": "second value"
        }
     },
     {
        "_index": "test",
        "_type": "el_test",
        "_id": "AVKmmYCI3xnXT_BGRA3S",
        "_score": 0.028130025,
        "_source": {
           "id": 0,
           "myField": "first value"
        }
     }
     ]
  }
  }

As I got a list of ElTest object back, I cannot retrieve the _score value (0.8838835 and 0.028130025). Is there a way to get the value in Spring-data?

I would imagine something like

@Document(indexName = "test", type = "el_test")
public static class ElTest{
    private long id;
    private String myField;
    @Score
    private double score;
}

Where Score would be an annotation indicating the field is filled by Elasticsearch.

The reason why I want the score is to sort the list in the gui.

Due to the many dto mappings from Elasticsearch up to the gui the list gets unsorted at some place. It is therefore not reliable.

I could of course add an score value by hand, but it would require to iterate over the list. Which is not very nice.

I use Spring-data-elasticsearch Version 1.2.0 and therefore Elasticsearch 1.4.4

Answer 1

I was looking for the same.

I have found that in ElasticsearchTemplate class instance of DefaultResultMapper is used for it:

@Override
public <T> FacetedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
    long totalHits = response.getHits().totalHits();
    List<T> results = new ArrayList<T>();
    for (SearchHit hit : response.getHits()) {
        if (hit != null) {
            T result = null;
            if (!Strings.isNullOrEmpty(hit.sourceAsString())) {
                result = mapEntity(hit.sourceAsString(), clazz);
            } else {
                result = mapEntity(hit.getFields().values(), clazz);
            }
            setPersistentEntityId(result, hit.getId(), clazz);
            results.add(result);
        }
    }
    List<FacetResult> facets = new ArrayList<FacetResult>();
    if (response.getFacets() != null) {
        for (Facet facet : response.getFacets()) {
            FacetResult facetResult = DefaultFacetMapper.parse(facet);
            if (facetResult != null) {
                facets.add(facetResult);
            }
        }
    }

    return new FacetedPageImpl<T>(results, pageable, totalHits, facets);
}

Score is stored inside SearchHit instance, however it is ignored in hit.sourceAsString()...

As a workaround I have created a simple interface:

/**
 * Object to have score from elastic search
 */
public interface Scoreable {

    float getScore();

    void setScore(float score);
}

and extended DefaultResultMapper:

/**
 * Results mapper with score support
 */
public class ScoreResultsMapper extends DefaultResultMapper {

    public ScoreResultsMapper(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext) {
        super(mappingContext);
    }

    @Override
    public <T> FacetedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
        FacetedPage<T> resultPage = super.mapResults(response, clazz, pageable);
        Iterator<T> it = resultPage.getContent().iterator();
        for (SearchHit hit : response.getHits()) {
            if (hit != null) {
                T next = it.next();
                if (next instanceof  Scoreable) {
                    ((Scoreable) next).setScore(hit.score());
                }
            }
        }
        return resultPage;
    }
}

Here I am just checking whether returned type is an instance of Scoreable and if so I am putting score into it

So now I can configure ElasticsearchTemplate using new mapper (I am using spring-boot):

@Bean
public ElasticsearchTemplate elasticsearchTemplate() {
    MappingElasticsearchConverter converter = new MappingElasticsearchConverter(new SimpleElasticsearchMappingContext());
    ScoreResultsMapper mapper = new ScoreResultsMapper(converter.getMappingContext());
    return new ElasticsearchTemplate(client(), converter, mapper);
}

Of course all my documents with score have to extend Scorable. It might be possible we have to overwrite some other methods of DefaultResultMapper to have score support in other types of queries.