TACO · Threedimensional Adaptive Camera with Object Detection and Foveation

TACO develops a 3D sensing system with real 3D foveation properties endowing service robots with a higher level of motion and affordance perception and interaction capabilities with respect to everyday objects and environments. By 3D foveation properties we mean properties based on the process of acquiring 3D images with coarse level of details, applying fast object recognition techniques to identify areas of interest in the coarse 3D image and then concentrate the image acquisition on details of interest allowing for higher resolution 3D sampling of these details. This new 3D foveation concept will allow robots to interact with everyday environments in a more natural and human-like manner, increasing the level of detail whenever needed for interaction between the robot and everyday objects and humans. These 3D foveation properties are achieved by utilising the power of micro-mirror MEMS technology combined with state-of-the-art time-of-flight methods to ensure a system that is small, light-weight and easily mounted on an ordinary-sized service robot or even a robot arm. The project will explore control strategies for 3D foveation allowing 3D robot vision that is adaptable with space- and time-variant sampling, processing and understanding. The project will verify and test the 3D sensing system in a robotic environment, exploring the capabilities of the system to allow the robot to navigate autonomously and interact with a diverse number of everyday objects. The TACO consortium has RTD partners and industrial end users: Fraunhofer IPMS (micro-mirror scanning device), Fraunhofer IPM (3D range sensors), CTR (Electronics), SINTEF (3D foveation software), TU Wien (benchmarking with state-of-the-art 3D sensing methods), Shadow Robotics (application towards robot grippers) and Oxford Technologies (application towards robots for harsh environments)