Skip to content

Deep Learning and Reinforcement Learning (refresher course)

PDF icon Download in PDF

  • Lecturer: Louis Annabi Occelli (annabi@isir.upmc.fr)
  • Course code:
  • Student workload: 14 h of lectures, 2 h of tutorials, 12 h of labs
  • Credits: 3 ECTS
  • Specialization tracks: ROBOT (elective) SI (elective) MeDH
  • Semester offered: S1 S2 S3 S4
  • Language of instruction: French English
  • Targeted audience: Eng. Sc. department Other : ENSAM and newly admitted students
  • Localization : PMC Campus Other :

Course Overview

This course unit trains students in deep learning and reinforcement learning. A first fundamental block focuses on deep learning (neural networks, backpropagation and standard architectures). A second block introduces reinforcement learning.

Mots-clés : Deep learning, reinforcement learning

Prerequisites

Students should have previously acquired the following prerequisites to follow this course:

  • Install and use a programming environment (language, IDE)
  • Prepare data for learning
  • Develop an appropriate experimental protocol to evaluate model performance

Intended Learning Outcomes

By the end of this course, students will be able to:

  1. Choose the paradigm suited to the task and data (supervised, unsupervised or reinforcement learning)
  2. Choose a neural-network architecture suited to the task, data and resources (MLP, RNN, CNN)
  3. Choose the learning process (loss function, optimization algorithm, initialization)
  4. Size a network (number of layers and neurons)
  5. Create a graphical representation of a neural network
  6. List an algorithm’s hyperparameters (for deep learning or reinforcement learning)
  7. Implement fundamental algorithms (gradient descent, backpropagation, dynamic programming, temporal-difference learning, etc.)
  8. Master a high-level deep-learning library (PyTorch or TensorFlow)
  9. Carry out the learning process defined during the design phase by optimizing the associated multiple loss functions
  10. Reproduce training processes and experimental results
  11. Monitor and interpret the results to ensure optimal learning

Indicative Teaching Sequence and Methods

Week C/TD/TP* Content Preparation Learn.\ outc.
S1 C1 Backpropagation, perceptron, MLP, activation function   1, 7
S2 TP1 Backpropagation implementation   7, 9, 10, 11
S3 C2 Optimizers, generalization, initialization, normalization, regularization   1, 3, 4, 6
S4 TP2 Optimizers, improving speed and generalization on a classification task   4, 8, 9, 10, 11
S5 C3 Inductive bias, CNN, RNN   1, 2, 4, 5, 6
S6 TP3 CNN, RNN   4, 8, 9, 10, 11
S7 C4 Loss functions, unsupervised learning, AE, VAE, GAN   1, 3, 5, 6
S8 TP4 AE and VAE   8, 9, 10, 11
S9 C5 MDP, dynamic programming   1, 6
S10 C6 Tabular RL, TD(0), Q-learning, SARSA   1, 6
S11 TD1 Tutorial on lectures C5 and C6   7
S12 TP5 Q-learning   7, 10, 11
S13 C7 Introduction to deep reinforcement learning, DQN, DDPG   1, 6
S14 TP6 DQN   7, 8, 10, 11
  • C/TD/TP respectively corresponds to lectures, tutorials and lab sessions.

Sequence of the unit

The course unit is organized into two successive sequences:

  • Sequence 1: deep learning

  • Sequence 2: reinforcement learning

Indicative Assessment of Intended Learning Outcomes (1st session)

Week Individ./group In-person/remote Type of exam Evaluated outcomes Scale %

2nde session

Session Individ./group In-person/remote Type of exam Evaluated outcomes Scale %

Logo SDI Date of generation of this unit description: 06/10/2026 Logo SDI