Seeing to Learn (S2l) : Observation learning in robotics (Oct 2017 – Present)

Seeing to learn

Seeing to learn (S2l) is an computer vision-robotics project currently ongoing at University of Leeds, which is aimed at developing advanced observation learning methods for robotics systems. It addresses the inability of current robotic systems to learn from human demonstrations. The project envisions a future where robots could acquire new skills by just observing humans perform a task or even by watching online tutorial videos of demonstrations. In future the robots equipped with these learning methods could be applied in several real world conditions ranging from home to work environments such as construction sites where it could learn to perform the relevant tasks of drilling holes, hammering nails or screwing a bolt just by observing other workers. [website] [report] [poster]


Self-Repairing Cities (Oct 2016 – Sep 2017)

'Self Repairing Cities' is a 5-year EPSRC granted £4.2m project aimed at exploring the use of robots and autonomous systems in city infrastructure. The team is formed of a consortium of the Universities of Leeds (lead), Birmingham, UCL and Southampton.
Seeing to learn My main role in the project was to explore the use of deep learning based computer vision methods for crack detection in pavement images. A special kind of neural networks called convolutional neural networks (CNNs) was used for this. The convolutional layers in the CNN were able to automatically extract relevant visual features from pavement images for detecting cracks by using its multi layered hierarchal network architecture. These features were further used by the fully connected layers in the CNN to effectively classify the input pavement image into cracked and non-cracked categories. The results of this work was published in International conference for Automation and Robotics in Construction (ISARC) 2017.
[website] [view paper] [view code]
As a part of the project, I also worked on developing a simple visual tracking system for tracking colour coded autonomous ground vehicle from a drone, as a part of our entry 'A mock-up earthmoving truck autonomous operation from drones' to the 'Robots for Resilient Infrastructure competition' held in Leeds in association with UK robotics week 2017. [view code]


Computer Vision Based System Drowsiness Detection System (Aug 2015 – May 2016)

Project descriptionThe aim of the project was to develop a computer vision system for drowsiness detection. The developed system uses a ordinary web camera and uses a computer vision algorithm to detect whether the subject is drowsy or active.The first stage of the developed system is a tracking algorithm that accurately tracks human eyes in the video of the subject taken using a normal consumer grade camera. Then it determines whether the eyes are closed or open using a combination of the HOG features and SVM classifiers. After this the system calculates the PERCLOS (time for which eyes were closed in 1 min) value. So if the value is above a particular threshold then the person is drowsy else is active. [blog] [paper 1] [paper 2] [paper 3] [view code]


CAMbot : Customer Assistance Mobile Manupulator Robot (Jul 2014 – Apr 2015)

Project descriptionThe project develops A Customer Assistance Mobile Manipulator Robot that can be used in super markets and big shopping malls .It assists customers by recommending products by facial, gender and emotion recognition using image processing techniques, finding the required products with the help of a user interface developed in C# , identifying commodities in shelves using a RFID tagging system, helping the customers to navigate through the store using buidin maps stored in the robotic memory and helping them to pick and place the commodities using its robotic arm. The project has an interdisciplinary nature and includes Image processing (Robotic vision), embedded system (Microcontrollers), Electrical components (Motors, Servo and drivers), Mechanical design and kinematics (Load- Motor torque relation & robotic arm), Electronic circuits (power supply circuits) and programming (C#, Matlab, python &embedded C). [website] [paper 1] [paper 2] [paper 3] [code]



Automatic sorting and grading system for mangos (May 2014 – Dec 2014)

The project was aimed at developing an automatic system for sorting and grading of mangos based on computer vision algorithms using a robotic arm. The application of this system is aimed to replace the existing manual based technique of sorting and grading used in India. The developed system speeds up the entire process considerably and is more accurate and efficient than the traditional techniques used. The project was funded under Technical Education Improvement Programme (TEQIP) Phase II , CUSAT.
[code] [paper] [dataset]