PatOlympics 2011
Tuesday 7 June, 9:30 - 12:30
This morning session is organized in cooperation with the PROMISE Network of Excellence. It will feature the second edition of the PatOlympics, a live prototype evaluation exercise that complements evaluation tracks like CLEF-IP and TREC-CHEM. In parallel to the PatOlympics competition, research groups and companies will demonstrate their latest prototypes.
Procedure
Registered athletes receive a document collection to index using their own systems in advance. On the day of the event, they can show their systems on their own machines or via the web. Selected patent information users – the referees – prepare questions for the participating teams. In 30 minutes slots the referees then use the systems of the competing teams. Based on this interaction and backed by the PatOlympics server, where results will be compared and correlated with the referees’ set of correct answers, the teams will be judged by the quantity in combination with the qualitative relevance of their search results. In addition referees will evaluate the interface design and the overall performance of the system.
As relevance judgments are indeed subjective and may change during the evaluation exercise, referees will be able to alter the set of correct results, to make it even more challenging. A live score board will round up the thrill and constantly show the status of each competitor. Winners will be awarded with medals for best results, best user interface and ChemAthlon / Cross-Lingual Retrieving Gold.
Participants
Cross Language Retrieval
- Manisha Verma, IIIT Hyderabad, India
- Parvaz Mahdabi, University of Lugano, CH
- Gérard Dupont, Cassidian, FR
- Wouter Alink, Spinque, NL
ChemAthlon
- Julien Gobeil, BiTem Group, University Hospitals of Geneva, CH
- Le Zhao, CMU, US
Referees
- Teresa Loughbrough (Unilever R&D, UK)
- Stephen Adams (Magister Ltd., UK)
- Jane List (Consultant, UK)
