FINE · Free-viewpoint Immersive Networked Experience

The Free-viewpoint Immersive Networked Experience (FINE) project will be focused on researching and developing a novel end-to-end architecture for the creation and delivery of a new form of live media content. FINE will introduce the concept of live free-viewpoint content which will provide rich and compelling immersive experiences by allowing remote viewers to place a virtual camera in a real live-action scene and move it freely in space and time, heightening their sense of presence and reality.Advances in the acquisition, processing, transmission and exhibition of live free-viewpoint content will be accomplished by focusing the research on four main areas:1.\tFast (real-time) and highly accurate algorithms for smooth view interpolation and photorealistic 3D reconstruction of live events from multiple, high quality live video streams.2.\tReal-time tracking and marker-less motion capture of multiple characters.3.\tNew coding and transmission technologies to allow the synchronized delivery of geometry, imagery and metadata to a wide variety of end-users through New Generation Networks.4.\tIntegration of the developed free-viewpoint technologies in a networked end-to-end architecture, and their validation in experimental productions.Several user centred scenarios will be defined for both professional and home users. These scenarios will serve as a framework to demonstrate and validate the developed technologies and their usefulness. FINE will have a large impact on several platforms: Internet, broadcast TV, interactive TV, mobile, online video games and digital cinema. Even though the results will be directly applicable to the media industry, they will also have clear benefits for other markets such as teleconferencing, medical imaging, surveillance and security. As part of the outcomes, a multi-view video dataset and a collection of open source tools to manage it will be built and released to the scientific community.