Course Description


The aim of this course is to understand how to build robot systems that can sense their environment to then perform an action. To do this, robots must have the ability to receive information from their environment but also an understanding of themselves within it. Importantly, how can we capture this information greatly influences our subsequent actions both in terms of the availability of actionable data but also the processing required to interpret. By exploring these trade-offs we will explore in depth the sensory systems used to capture information and in turn develop models that enable robots to cooperate with its environment in different ways.

In this course, we will discuss a range of sensing and cognitive control strategies. This theory will then be used to design robotic systems to perform manipulation tasks to cope with unstructured environments such as localizing objects for grasping.

Learning Outcomes

  • Understand the main concepts related to robotic manipulation and sensing.
  • Develop methods for tackling uncertainty in robotic manipulation systems.
  • Read scientific literature in robotics to choose approaches for a particular problem.
  • Implement state-of-the-art algorithms on simulated manipulators and sensors.

Theory Prerequisites


  • Representing Poses and Kinematics for Robot Manipulation
  • Visual & other Exteroceptive Sensing
  • Visual pose estimation under uncertainty
  • Force/Torque & other Proprioceptive Sensing
  • Sensing-based Grasping
  • Pick-and-Place Methods
  • Navigation Among Movable Objects (NAMO)
  • Human-Robot Interaction and Collaboration
  • Reinforcement Learning for Grasping

Syllabus Structure

  • One 2-hours lecture will discuss background theory.
  • One 1-hour lecture for research reading and presentation from the students.
  • One 1-hour lab session will focus on simulated experiments on manipulators and sensors for grasping purposes.

Coding Prerequisites

  • Previous programming experience: C++, ROS, PCL.
  • Motivation to work hard.
  • An introductory course on ROS from ETH can be found here.



The Moodle page for the course: here.

Methods of assessment

  • Coursework 0 (CW0): 0%
  • Coursework 1 (CW1): 20%
  • Coursework 2 (CW2): 35%
  • Coursework 3 (CW2): 45%

Working Groups

Unless specified differently, the research presentations, practical sessions, homework assignments, and final project will be done by students that they form groups (not necessary the same groups for every category). The pairs will be assigned randomly and based only on some requirements such that at least every group has an Ubuntu 18.04LTS, ROS Melodic system to work on.

  • Total number of students: 52
  • Research Presentation: 3-4 students (total: 14 teams)
  • Coding HW: 3 students (total: 17 teams)
  • Research Report: 3 students (total: 17 teams)
  • Project and Report: 3 students (total: 17 teams)

Academic Integrity

The UCL academic integrity policy applies to your work in this course for: written homework, coding work, and coding assignments. Cheating and other acts of academic dishonesty will be referred to the corresponding UCL office: here.

Instruction Staff


  1. Dimitrios Kanoulas - 60% (
    Office hours: TBA, 09:00-10:00am, Zoom (link).
  2. Francisco Vasconcelos - 40% (
    Office hours (Dimitrios): TBA, 09:00-10:00am, Zoom (link).


  1. Denis Hadjivelichkov (, PhD Student in Robotics and Machine Learning
  2. Luke Beddow: (, PhD Student in Robotics and Mechatronics
  3. Maria Stamatopoulou: (, PhD Student in Robotics and Machine Learning
  4. Sicelukwanda Zwane (, PhD Student in Robotics and Machine Learning