L3TD · Learn to learn human learning process from teleoperated demonstrations

Learning from demonstration (LfD) is a paradigm for enabling robots to autonomously learn from demos to perform new tasks. But, environmental changes, expensive demonstration cost, and potential uncertainties caused by data-based learning make it hard to be applied in actual. The project aims to propose a robot skill learning framework from human learning process via a teleoperation interface to achieve human-like skill learning characteristics such as few-shot learning, learning from failed attempts and tentative actions, and strong skill transfer and generalization ability. Five work packages will be taken to realize the objectives. First, a teleoperated interface will be equipped with multi-sensors and special exoskeleton to minimize information difference between humans and robots. After building a scalable primitive skill (PS) library based on task segmentation with multimodal information, new theories of PS learning and PS-based task graph learning are explored. PS will be learned and generalized based on improved meta-learning that is associated and explained by physical laws and neural motor disciplines. The PS-based task graph will be learned from the human learning process, achieving failure reasoning and adaptation to zero/few-shot tasks. Some practical problems e.g. incomplete data set and difference of sim-to-real applications will also be addressed. Finally, the previous theories will be certified by medical robot tasks. The applicant will acquire a solid state-of-the-art interdisciplinary scientific training in the multidisciplinary research fields, such as artificial intelligence, robotics technologies and mechanical design, and that will enable him to generate new scientific knowledge and quickly develop his research career and leadership. The final aim is to consolidate Europe as the world leader in robot and AI areas and to benefit European robotics applications in industry, surgery, and nuclear waste disposition.