The main objective of this course is to learn how Reinforcement Learning (RL) solutions help solve real-world problems through trial-and-error interaction by implementing a complete RL solution from beginning to end.

In order to take the course without excessive difficulty, it is recommended to have previously acquired the following skills:

• Python: Basic knowledge

• Programming level: Data structures and algorithms

• Statistics: Conditional probability

• Machine Learning: Basic knowledge of supervised classification and neural networks

It is also recommended to have taken or be taking the Deep Learning subject.

This course provides the basic concepts of reinforcement learning. It gives students a detailed understanding of various topics: Markov Decision Processes, sample-based learning algorithms and deep reinforcement learning.

Topic 1 Introduction to the course

Topic 2 Introduction to Reinforcement Learning: definition of basic concepts such as Markov Decision Proccess and Value Functions

Topic 3 Dynamic Programming: methods to solve the problem when the model is known: policy iteration and value iteration methods

Topic 4 Monte Carlo Methods: methods to solve the problem learning from simulated experiences.

Topic 5 Temporal-Difference Learning: combination of Dynamic Programming and Monte-Carlo: SARSA, Q-Learning and variants

Topic 6 Deep Reinforcement Learning: Function approximation, Batch Learning, Deep Q-Network and Rainbow (combination of several improvements in Deep Reinforcement Learning)

Master classes, seminars, laboratories, assignments, practices and presentations.



The skills and competences demonstrated in all aspects of the subject make up your note: active participation, tasks, practice, presentations, etc.

• Final Assessment System
• Tools and qualification percentages:
  • The skills and competences demonstrated in all aspects of the subject make up their note: active participation, individual tasks, group practices, presentations, etc. (%): 100

The subject has two possible evaluation modes: final and continuous.



The continuous evaluation is the preferred mode. It establishes a set of activities that allows assessing the progress of each student throughout the course. Thus, the continuous evaluation is offered by default to students who should deliver the assignments of the subject in the established framework including assistance, presentations and face-to-face activities.



Students can also be evaluated through the final evaluation mode. In this case, the students on established dates (when reaching around 60% and 80% of the course) must submit to the teaching supervisors a formal resignation to the continuous evaluation. Then, the teaching supervisors will assign a mandatory practical work and a date for an oral presentation prior to the date indicated for ordinary and extraordinary examination.



The weight of the different aspects to consider in the two alternative forms of evaluation is presented below.



Continuous Evaluation



• 3 obligatory assignments (100%), 40% of the mark must be obtained in each one in order to pass subject

◦ Individual Multiple Choice Exam: 40%

◦ Oral presentation in group (3-4 people) about a Reinforcement Learning applied paper: 30%

◦ Practical work in group (3-4 people): 30%



Final Evaluation



Delivery of mandatory practical work and oral presentation prior to the written exam on the date indicated for the ordinary and extraordinary examination: 100%

Final Evaluation



Delivery of mandatory practical work and oral presentation prior to the written exam on the date indicated for the ordinary and extraordinary examination: 100%

• eGela
• Google Colab

Basic bibliography

Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction 2nd Edition, 2018

In-depth bibliography

Maxim Lapan. Deep Reinforcement Learning Hands-on. Packt Publishing Ltd., 2nd edition, 2020.

Web addresses

Artificial Intelligence. Elsevier Science.