Teams for PatOlympics announced

2010 | 02 | 12

To foster the interaction between system developers and information professionals we invite you to experience an innovative and very athletic competition:

The PatOlympics Protoype Evaluation Exercise perfectly complements laboratory-style evaluation campaigns such as TREC-CHEM and CLEF-IP. 

This half-day event will provide an interactive evaluation platform where prototypes of state-of-the-art patent retrieval systems will be tested and driven to their limits by patent retrieval professionals. Participants will be able to interact with the experts, demonstrate the power and flexibility of their prototype and compete with other participants for the medals of the PatOlympics.

The 2010 PatOlympics include two PatSports: ChemAthlon and Cross-Language Retrieving. Participating groups can register to either one or both of the sports.


Participants 

  • Charles Jochim, University of Stuttgart
  • Santiago Correa García, University Politécnica de Valencia
  • Wouter Alink, Spinque
  • Patrice Lopez, Humboldt University
  • Julien Gobeill , University of Applied Sciences, Geneva
  • Manisha Verma, International Institute of Information Technology, Hyderabad


Procedure

Registered athletes receive a document collection to index using their own systems in advance. On the day of the event, they can then show their systems on their own machines. Selected patent information users – the referees – prepare questions for the participating teams. In 30 minutes slots the referees then use the systems of the competing teams. Based on this interaction and backed by the PatOlympics server, where results will be compared and correlated with the referees’ set of correct answers, the teams will be judged by the quantity in combination with the qualitative relevance of their search results. In addition referees will evaluate the interface design and the overall performance of the system.

As relevance judgements are indeed subjective and may change during the evaluation exercise, referees will be able to alter the set of correct results, to make it even more challenging. A live score board will round up the thrill constantly showing the status of each competitor. Winners will be awarded with medals for best results, best user interface and ChemAthlon / Cross-Lingual Retrieving Gold.


If you would like to participate as a referee, please contact m.lupu@ir-facility.org.