CODEWITHSUNDEEP
search-engineinformation-retrievaldata-retrieval
John M
John M

Reputation: 217

How does trec_eval calculates Mean Average Precision (MAP)?

I'm using TREC_EVAL to evaluate a search engine. I'd like to know how it calculates the Mean Average Precision (MAP). I'm sure it doesn't calculate a simple average of the average precisions (AP). It seems a weighted arithmetic but I can't understand which weights are used.

Upvotes: 1

Views: 1044

Answers (1)

Ellen Voorhees
Ellen Voorhees

Reputation: 241

MAP is indeed a simple arithmetic mean of the AP scores for individual topics. (But remember that AP for an individual topic is computed over all relevant documents. There is a frequently used incorrect definition of AP that computes 'AP' over only relevant retrieved documents, but that is a nonsensical measure as it rewards retrieving fewer relevant.)

The "Common Measures" section of the Appendix to each TREC proceedings has the definition of the most commonly used TREC eval measures. TREC proceedings are in the Publications section of the TREC web site, trec.nist.gov.

Ellen Voorhees TREC project manager NIST

Upvotes: 4

Related Questions