Skip to content

Perception for Robotics

PDF icon Download in PDF

  • Lecturer: Xavier Clady & Anis Sabani ()
  • Course code:
  • Student workload: 12 h of lectures, 16 h of labs
  • Credits: 3 ECTS
  • Specialization tracks: ROBOT (elective) SI MeDH
  • Semester offered: S1 S2 S3 S4
  • Language of instruction: French English
  • Targeted audience: Eng. Sc. department Other :
  • Localization : PMC Campus Other :

Course Overview

To perform autonomous actions, a robot needs advanced artificial-perception systems. This course introduces students to methodologies ranging from conventional to recent approaches. One of the main modalities used—vision sensors—is studied in depth because it is both highly complex and highly informative. Other modalities, including active sensors such as scanning lasers and proprioceptive sensors such as IMUs, are also considered, particularly through multisource and multisensor data fusion. The course examines theoretical foundations such as vision-sensor geometry for metrology and the extraction and representation of information for pattern recognition. Some topics review concepts studied in depth in other Robotics courses, while others provide extensions. The emphasis is on practical implementation in high-level software architectures for essential autonomous-robotics applications, including detection, localization, tracking and identification of objects, places, people, actions and behaviors. Special attention is given to odometry as a building block for mobile-robotics applications such as localization, 2D/3D mapping and SLAM. Overall, students acquire the theoretical and practical knowledge needed to select and implement a perceptual system, strategy and architecture in an autonomous robot, from modeling through use in a given task.

Mots-clés :

Prerequisites

Students should have previously acquired the following prerequisites to follow this course:

  • Fundamentals of image processing (2D convolution, filters, etc.) — see Image1 course unit
  • Fundamentals of signal and audio processing (1D convolution, filters, etc.) — see Son1 course unit
  • Fundamentals of state-space representation (equation of motion)
  • Probability and estimation (see Estimation Theory and Parametric Identification course unit)
  • Algebra (matrix calculations)

Intended Learning Outcomes

By the end of this course, students will be able to:

  1. Understand fundamental image and video concepts frequently used in robotics: extraction (advanced image or video segmentation, detection and characterization of 2D/2D+t/3D points of interest, etc.), representation of visual information (appearance models, bag of visual words, etc.), and robotics applications (detection, localization and/or recognition of places, people, objects, actions and behaviors).
  2. Apply fundamental computer-vision concepts: projective geometry, simple camera models and calibration, stereoscopic/3D vision and visual odometry.
  3. Understand conventional strategies for multimodal data fusion and their applications in robotics, particularly with exteroceptive sensors (camera, microphones, laser, etc.) and proprioceptive sensors (IMU).
  4. Implement a visual-odometry system using projective geometry
  5. Integrate a perceptual system, strategy and architecture using multimodal data (camera, laser, IMU) for incremental localization of a mobile robot

Indicative Teaching Sequence and Methods

Week C/TD/TP* Content Preparation Learn.\ outc.
S1 CM1-2 (2x2h) Sensors in robotics, multisensor fusion AAV3
S2 CM3-4 (2X2h) Camera models and calibration, projective geometry AAV2
S3 CM-6 (2X2h) Stereo vision, points of interest, odometry AAV1, AAV2, AAV3
S4 TP1 (4h) Visual odometry for a flying drone using projective geometry (Part 1) AAV1-4
S5 TP2 (4h) Visual odometry for a flying drone using projective geometry (Part 2) AAV1-4
S6 TP3 (4h) Sensor fusion for localization of wheeled mobile robots (Part 1) AAV1-3, AAV5
S7 TP4 (4h) Sensor fusion for localization of wheeled mobile robots (Part 2) AAV1-3, AAV5
  • C/TD/TP respectively corresponds to lectures, tutorials and lab sessions.

Indicative Assessment of Intended Learning Outcomes (1st session)

Week Individ./group In-person/remote Type of exam Evaluated outcomes Scale %
S4 Individual In-person Quiz AAV1-3 30%
S8 Individual In-person Written exam AAV1-5 70%

2nde session

Session Individ./group In-person/remote Type of exam Evaluated outcomes Scale %

Bibliographic references

  • Artificial Intelligence : A modern Approach, S. Russel and P. Norvig, Prentice Hall, 1995.
  • Multiple view geometry, Richard Hartley and Andrew Zisserman, Cambridge University Press
  • Local invariant feature detectors: A survey. T. Tuytelaars and K. Mikolajczyk, Foundations and Trends in Computer Graphics and Vision, vol. 3, no. 3, pp. 177–280, 2007.
  • Kalman Filter for Robot Vision: A Survey, Chen, S.Y., IEEE Transactions on Industrial Electronics, Volume: 59 , Issue: 11, pp 4409 - 4420, Nov. 2012
  • A comprehensive review of current local features for computer vision, J. Li and N. M. Allinson, Neurocomputing, vol. 71, pp. 1771–1787, 2008.

Logo SDI Date of generation of this unit description: 06/10/2026 Logo SDI