Skip to content

Deep Learning and introduction to Reinforcement Learning

PDF icon Download in PDF

  • Lecturer: Louis Annabi (annabi@isir.upmc.fr)
  • Course code: UM4RBI20-DL
  • Student workload: 14h of lectures, 2h of tutorials, 12h of labs
  • Credits: 3 ECTS
  • Specialization tracks:
  • Semester offered: S1 S2 S3 S4
  • Language of instruction: French English
  • Targeted audience: Eng. Sc. department Other :
  • Localization : PMC Campus Other :

Course Overview

This course aims to train students in machine learning with a focus on deep learning. The first fundamental block focuses on deep learning (neural networks, backpropagation, classical architectures). A second block offers an introduction to two other learning paradigms: reinforcement learning and unsupervised and self-supervised learning.

Mots-clés : Deep learning. Classical neural architectures: CNN, RNN, Autoencoders. Introduction to reinforcement learning.

Prerequisites

Students should have previously acquired the following prerequisites to follow this course:

  • Mathematics: Linear algebra, vector and matrix calculus, numerical optimization of continuous functions with multiple variables, differential calculus, probability theory.
  • Computer science: Install and use a programming environment (language, IDE), use a virtual environment and code version management software.
  • Machine learning: Understand the fundamental concepts and methods of machine learning. Prepare data for learning. Develop an appropriate experimental protocol to evaluate model performance.

Intended Learning Outcomes

By the end of this course, students will be able to:

  1. C1.2. Analyze a problem: Choose a neural network architecture suited to the task, data, and resources (MLP, RNN, CNN).
  2. C1.3. Analyzing a problem: Graphically representing a neural network.
  3. C2.1. Design: Implement fundamental algorithms (gradient descent, backpropagation, dynamic programming, temporal difference learning, etc.)
  4. C2.3. Design: Specify meta-parameters to optimize network parameter learning
  5. C2.4. Design: Select the learning process (cost function, optimization algorithm, initialization)
  6. C3.2. Implementation: Mastering a high-level library dedicated to deep learning (PyTorch or TensorFlow)
  7. C3.4. Implementation: Achieve the learning defined in the design phase by optimizing multiple associated loss functions.
  8. C3.5. Implementation: Reproduce learning and experimental results
  9. C4.2. Evaluation: Check and interpret the results obtained to ensure optimal learning.
  10. C4.3. Evaluation: Diagnose the causes of suboptimal network behavior in order to propose solutions to remedy it.

Indicative Teaching Sequence and Methods

Week C/TD/TP* Content Preparation Learn.\ outc.
S1 C1 + C2 + TP1 Backpropagation, introduction to DL Practical work on course C1 2, 3, 5, 6
S2 C3 + TP2 + TP3 Convolutional neural networks Practical assignment on course C2 Practical assignment on course C3 1, 2, 4, 6, 8, 9, 10
S3 C4 + TP4 Recurrent neural networks Practical assignment based on lecture C4 1, 2, 4, 6, 8, 9, 10
S4 C5 + TP5 Introduction to tabular RL Practical assignment based on lesson C5 3, 7, 9, 10
S5 C6 + TP6 Introduction to deep RL Practical assignment based on lecture C6 3, 5, 6, 7, 9, 10
S6 C7 + TP7 Unsupervised and self-supervised learning Practical assignment based on course C7 1, 2, 3, 4, 5, 6, 7, 9, 10
  • C/TD/TP respectively corresponds to lectures, tutorials and lab sessions.

Sequence of the unit

The unit is divided into two successive sequences:

  • Sequence 1: deep learning (neural networks, backpropagation, classical architectures) [C1-4, TP1-4, TD1-2, Quiz 1-2]

  • Sequence 2: other paradigms (reinforcement learning and unsupervised and self-supervised learning) [C5-7, TP5-6, Quiz 3]

Indicative Assessment of Intended Learning Outcomes (1st session)

Week Individ./group In-person/remote Type of exam Evaluated outcomes Scale %
S3 Individual In-person Quiz 2, 3, 5 6%
S5 Individual In-person Quiz 1, 2, 3, 4, 5 7%
S7 Individual In-person Quiz 1, 3, 5 7%
S5 Individual In-person Labs 4, 5, 6, 7, 8, 9, 10 40%
S8 Individual In-person Written 1, 2, 3, 4, 5, 9, 10 40%

2nde session

Session Individ./group In-person/remote Type of exam Evaluated outcomes Scale %
2 Individual In-person Written 60%
1 Individual In-person Labs 4, 5, 6, 7, 8, 9, 10 40%

Logo SDI Date of generation of this unit description: 14/01/2026 Logo SDI