[Asis-l] CLEF Lab - Call for Participation - Living Labs for IR Evaluation (LL4IR)

Liadh Kelly liadh.kelly at gmail.com
Tue Mar 10 11:09:30 EDT 2015


The Living Labs for Information Retrieval Evaluation (LL4IR) CLEF Lab

September, 2015, Toulouse, France


Register at:

[Registration closes: 30 April, 2015]


This lab provides a benchmarking platform for researchers to evaluate their
ranking systems in a live setting with real users in their natural task
environments. The lab acts as a proxy between commercial organizations
(live environments) and lab participants (experimental systems).

The lab focuses on three use-cases (using a shared API):

Task 1: product search (on the REGIO JÁTÉK e-commerce site)
Task 2: local domain search (on the University of Amsterdam’s website)
Task 3: web search (through Seznam, a major commercial web search engine)


What is in it for you as a lab participant?


- Access to privileged commercial data (click through, etc.).
- Access to new static TREC-style collections.
- Access to the living labs evaluation environments.
- Opportunity to test your IR systems on real live systems with real live


Current Important dates


31 Mar, 2015:  Training period ends

1-15 Apr, 2015:  Uploading test runs

15 Apr, 2015:  Testing period begins

15 May, 2015:  Testing period ends


Further Information


Further information is available on the lab website (
http://living-labs.net/clef-lab/) or by emailing the lab organisers.


Lab Organisers


Krisztian Balog – University of Stavanger, Norway (krisztian.balog (at)

Liadh Kelly – Trinity College Dublin, Ireland (liadh.kelly (at) tcd.ie)

Anne Schuth – University of Amsterdam, The Netherlands (anne.schuth (at)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/asis-l/attachments/20150310/f31062c7/attachment.html>

More information about the Asis-l mailing list