CODEWITHSUNDEEP
elasticsearchlucene
Datageek
Datageek

Reputation: 26679

How to normalize ElasticSearch scores?

For my project I need to find out which results of the searches are considered "good" matches. Currently, the scores vary wildly depending on the query, hence the need to normalize them somehow. Normalizing the scores would allow to select the results above a given threshold.

I found couple solutions for Lucene:

How would I go ahead and apply the same technique to ElasticSearch? Or perhaps there is already a solution that works with ES for score normalization?

Upvotes: 4

Views: 5646

Answers (2)

Bikas Katwal
Bikas Katwal

Reputation: 2035

It's a bit late. We needed to normalise the ES score for one of our use cases. So, we wrote a plugin that overrides the ES Rescorer feature.

Supports min-max and z score.

Github: https://github.com/bkatwal/elasticsearch-score-normalizer

Usage: Min-max

{
  "query": {
    ... some query
  },
  "from" : 0,
  "size" : 50,
  "rescore" : {
      "score_normalizer" : {
        "normalizer_type" : "min_max",
        "min_score" : 1,
        "max_score" : 10
      }
   }
}

Usage z-score:


  "query": {
    ... some query
  },
  "from" : 0,
  "size" : 50,
  "rescore" : {
      "score_normalizer" : {
        "normalizer_type" : "z_score",
        "min_score" : 1,
        "factor" : 0.6,
        "factor_mode" : "increase_by_percent"
      }
   }
}

For complete documentation check the Github repository.

Upvotes: 0

Genapshot
Genapshot

Reputation: 41

As far as I searched, there is no way to get a normalized score out of elastic. You will have to hack it by making two queries. First will be a pilot query (preferably with size 1, but rest all attributes same) and it will fetch you the max_score. Then you can shoot your actual query and use functional_score to normalize the score. Pass the max_score you got as part of the pilot query in params to function_score and use it to normalize every score. Refer: This article snippet

Upvotes: 3

Related Questions