MindSee · Symbiotic Mind Computer Interaction for Information Seeking

While real time analysis of user states based on signals from the brain and peripheral physiology have made progress (as separate branches of research), the possibility to predict intentions and infer implicit user variables is still a grand challenge in real-world applications. As a novel approach, MindSee proposes to fuse EEG as a main sensor with peripheral physiological sensors (EDR, fEMG, eye gaze and pupillometry) and context information for an unobtrusive acquisition of implicit measures of perception, cognition and emotions. Real-time estimates of these hidden user states will be exploited to complement keyboard and gestural input exemplary in the real-world application of scientific literature search where the information exploration of the user is guided by a co-adaptation with the computer. The proposed Symbiotic Information Seeking System will provide a wide range of visualization resources adapting information retrieved and its relevance, cognitive ergonomic complexity, and aesthetic aspects.For the target application of scientific literature search, MindSee builds upon a cutting-edge retrieval system that has access to 50 million documents from the main scientific databases. MindSee will be developed in an iterative approach with three full cycles of implementation and evaluation of increasingly complex symbiotic interactions in information seeking. The iterative evaluation of MindSee technology and validation of underlying methods will be done in realistic user experiments with user groups of different skills and areas.The MindSee project will develop a new symbiotic information retrieval system capable of more than doubling the performance of information seeking in realistic tasks, compared to mainstream tools. MindSee symbiotic interaction will deliver solutions to increase productivity and creative potential. Several MindSee results are exploitable and applicable to other information seeking contexts beyond scientific literature search!