CODEWITHSUNDEEP
information-retrieval
user3490622
user3490622

Reputation: 1001

trec_eval: does iprec_at_recall staying constant mean something is wrong?

I am completely new to trec_eval. When I run it for a set of queries, I get the following results for iprec_at_recall:

iprec_at_recall_0.00    all     0.5059
iprec_at_recall_0.10    all     0.5059
iprec_at_recall_0.20    all     0.5059
iprec_at_recall_0.30    all     0.5059
iprec_at_recall_0.40    all     0.5059
iprec_at_recall_0.50    all     0.5059
iprec_at_recall_0.60    all     0.5059
iprec_at_recall_0.70    all     0.5059
iprec_at_recall_0.80    all     0.5059
iprec_at_recall_0.90    all     0.5059
iprec_at_recall_1.00    all     0.5059

So my precision is not changing as a function of recall thresholds. Does this necessarily imply a problem with my data?

Upvotes: 0

Views: 765

Answers (1)

Ellen Voorhees
Ellen Voorhees

Reputation: 241

iprec_at_recall_X is the measure for interpolated precision at standard recall level X. The particular rule used to interpolate precision at standard recall level X in trec_eval is to use the maximum precision obtained for the query for any actual recall level greater than or equal to X (this is how there can be a precision value for recall level 0). You can read more about how trec_eval computes measures in the appendix to (some of) the TREC proceedings, for example, see https://trec.nist.gov/pubs/trec20/appendices/measures.pdf .

So, my guess is that you are using a very small collection (or, at least, one with very few relevant documents) such that you reach 100% recall very early in your ranked list.

Ellen Voorhees

TREC project manager

NIST

Upvotes: 2

Related Questions