Information theory
Download in PDF- Lecturer: Nicolas OBIN (nicolas.obin@sorbonne-universite.fr)
- Course code: UM4EET10-TI
- Student workload: 10h of lectures, 12h of tutorials, 6h of labs
- Credits: 3 ECTS
- Specialization tracks:
- Semester offered: S1 S2 S3 S4
- Language of instruction: French English
- Targeted audience: Eng. Sc. department Other :
- Localization : PMC Campus Other :
Course Overview
This course introduces students to the fundamentals of information theory. Starting with the definition of key concepts and mathematical definitions of quantitative information measurement, the course presents all the elements of an information processing chain: source coding and entropy compression, channel coding, linear and convolutional block error-correcting codes, symmetric and asymmetric encryption.
Mots-clés : Information measurement, coding, transmission, encryption, Shannon entropy, mutual information
Prerequisites
Students should have previously acquired the following prerequisites to follow this course:
- Mathematics #1: Linear algebra, vector and matrix calculus
- Mathematics #2: Random signal processing, random variables, probabilities, mathematical expectation
- Computer science: Programming (using Matlab), algorithms
Intended Learning Outcomes
By the end of this course, students will be able to:
- Calculate information metrics
- Encode a source with a fixed-size code
- Encoding a source with variable-length code and average coding length
- Calculate the source coding limits
- Calculate the capacity of a transmission channel
- Calculate the transmission limits of a source on a transmission channel
- Implement error correction upon receiving a noisy source
- Apply encryption and decryption to a source
Indicative Teaching Sequence and Methods
| Week | C/TD/TP* | Content | Preparation | Learn.\ outc. |
|---|---|---|---|---|
| S1 | C1 / TD1 | Quantitative measurement of information and source coding theorem | 1, 2 | |
| S2 | C2 / TD2 | Source coding, Huffman algorithm, channel capacity | 3, 4, 5, 6 | |
| S3 | C3 / TD3 | Corrective codes: linear block codes | 7 | |
| S4 | TD4 | Preparation for practical work | Exam1 (1h30) the same week | 1-7 |
| S5 | C4 / TD5 | Correction codes: convolutional codes | 7 | |
| S6 | C5 / TD6 | Information encryption: Symmetric algorithms, asymmetric algorithms | 8 | |
| S7 | TP1 | Simulation of a complete information processing chain in Simulink | 1-8 | |
| S8 | TP2 | Simulation of a complete information processing chain in Simulink | 1-8 |
- C/TD/TP respectively corresponds to lectures, tutorials and lab sessions.
Sequence of the unit
The unit is divided into three successive sequences:
-
Sequence 1: information measurement, source coding, and channel capacity (C1/2, TD1/2/3, and CC1)
-
Sequence 2: error correction codes and encryption (C3/4/5, TD4/5/6, and CC2)
-
Sequence 3: practical work
Indicative Assessment of Intended Learning Outcomes (1st session)
| Week | Individ./group | In-person/remote | Type of exam | Evaluated outcomes | Scale % |
|---|---|---|---|---|---|
| S4 | Individual | In-person | Written | 1-6 | 30% |
| S7-8 | Individual | In-person | Practical | 1-8 | 30% |
| S9 | Collective | In-person | Written | 7-8 | 40% |
2nde session
| Session | Individ./group | In-person/remote | Type of exam | Evaluated outcomes | Scale % |
|---|---|---|---|---|---|
| 2 | Individual | In-person | Written | 1-8 | 70% |
| 1 | Individual | In-person | Practical | 1-8 | 30% |
Bibliographic references
- C. Shannon, W. Weaver. Théorie Mathématique de la Communication, 1948
- O. Rioul. Théorie de l'information et du codage, Lavoisier, 2007.
Date of generation of this unit description: 14/01/2026
