PROMISE · Participative Research labOratory for Multimedia and Multilingual Information Systems Evaluation

Measuring is a key to scientific progress. This is particularly true for research concerning complex systems, whether natural or human-built. Multilingual and multimedia information systems are increasingly complex: they need to satisfy diverse user needs and support challenging tasks. Their development calls for proper evaluation methodologies to ensure that they meet the expected user requirements and provide the desired effectiveness.Large-scale worldwide experimental evaluations provide fundamental contributions to the advancement of state-of-the-art techniques through common evaluation procedures, regular and systematic evaluation cycles, comparison and benchmarking of the adopted approaches, and spreading of knowledge. In the process, vast amounts of experimental data are generated that beg for analysis tools to enable interpretation and thereby facilitate scientific and technological progress.PROMISE will provide a virtual laboratory for conducting participative research and experimentation to carry out, advance and bring automation into the evaluation and benchmarking of such complex information systems, by facilitating management and offering access, curation, preservation, re-use, analysis, visualization, and mining of the collected experimental data. PROMISE will: foster the adoption of regular experimental evaluation activities; bring automation into the experimental evaluation process; promote collaboration and re-use over the acquired knowledge-base; stimulate knowledge transfer and uptake.Europe is unique: a powerful economic community that politically and culturally strives for equality in its languages and an appreciation of diversity in its citizens. New Internet paradigms are continually extending the media and the task where multiple language based interaction must be supported. PROMISE will direct a world-wide research community to track these changes and deliver solutions so that Europe can achieve one of its most cherished goals.