CODEWITHSUNDEEP
pythonwhoosh
Haj Ayed Amir
Haj Ayed Amir

Reputation: 197

How to create a Whoosh custom weight function?

I want to create a function that combines two approaches for ranking results. The first is based on the position and the second is BM25F. In whoosh documentation, I found out that I can pass scoring.FunctionWeighting(custom_function) as a keyword argument to my searcher. My custom function is : 

def position_weight(searcher, fieldname, text, matcher):
    poses = matcher.value_as("positions")
    w1 = 1.0 / (poses[0] + 1)
    # w2 = BM25F(matcher) ? This is what I don't know how to do.
    return 0.25 * w1 + 0.75 * w2

I got this function from documentation, so I don't understand the arguments of this function.

Upvotes: 2

Views: 568

Answers (1)

Daniel Gonçalves
Daniel Gonçalves

Reputation: 1

Here's an example of how you could combine multiple scoring functions based on the same logic:

from whoosh.qparser import *
from whoosh import scoring
 
def custom_scoring(searcher, fieldname, text, matcher):
    frequency = scoring.Frequency().scorer(searcher, fieldname, text).score(matcher)
    tfidf =  scoring.TF_IDF().scorer(searcher, fieldname, text).score(matcher)
    bm25 = scoring.BM25F().scorer(searcher, fieldname, text).score(matcher)
    return frequency + tfidf + bm25

my_weighting = scoring.FunctionWeighting(custom_scoring)
with ix.searcher(weighting=my_weighting) as searcher:
    qp = QueryParser("content",ix.schema,group=OrGroup).parse(query)
    results = searcher.search(qp)

In further detail, the following are the explanations behind each argument:

  • searcher - is an object from the Searcher class wrapping an IndexReader;
  • fieldname - is the name of the schema field where you perform the searches;
  • text - the query text;
  • matcher - is the Matcher object through which you can find the documents matching your criteria.

Upvotes: 0

Related Questions