Seeing to learn (S2l) is an computer vision-robotics project currently ongoing at University of Leeds, which is aimed at developing observation learning methods for robotics systems. It addresses the inability of current robotic systems to learn from human demonstrations. The project envisions a future where robots could acquire new skills by just observing humans perform a task or even by watching online tutorial videos of demonstrations. In future the robots equipped with these learning methods could be applied in several real world conditions ranging from home to work environments such as construction sites where it could learn to perform the relevant tasks of drilling holes, hammering nails or screwing a bolt just by observing other workers.

Publications

[1] Leo Pauly, Defining the problem of observation Learning, Technical report, 2017. [report]
[2]Leo Pauly, Wisdom C. Agboh, Mohamed Abdellatif, David C. Hogg, Raul Fuentes “One-Shot Observation Learning”, IROS 2018 Workshop: Examining Sensing Modalities for Robust and Dextrous Object Manipulation, Poster, 2018.[poster]

Team

@leopauly
Leo Pauly
(2017-present)
PhD Student
Univeristy of Leeds
@leopauly
Mohammad Kaykanloo
(Nov - Dec 2017)
Msc student
Univeristy of Leeds
@Prof.David Hogg
Prof.David Hogg
Professor of Artificial Intelligence
Univeristy of Leeds
@Dr.Raul Fuentes
Dr.Raul Fuentes
Associate Professor
Univeristy of Leeds

Downloads

The code and datasets being used in this project will be made available soon.