Method
–Use Ad-hoc track data for
TREC 3, 6, 7, 8
–Systems search the SAME
Database
–Automatic Short Runs
–50 Topics and 1,000
Documents per topic è 50,000
documents
–Retrieval systems can submit
multiple runs
– è Select Best Run based Mean Average
Precision
– TREC
3 19 systems
928,709 documents found
– TREC
6 24 systems 1,192,557
documents found
– TREC
7 28 systems 1,327,166
documents found
– TREC
8 35 systems 1,723,929
documents found
–Compute Average by
summing over all 50 topics and divide by 50