Breadcrumb

Subject

XSL Content

Back

Deep Learning

General details of the subject

Mode
Face-to-face degree course
Language
English

Description and contextualization of the subject

Deep Learning neural network models have been successfully applied to natural language processing, and are now changing radically how we interact with machines (Siri, Amazon Alexa, Google Home, Skype translator, Google Translate, or the Google search engine). These models are able to infer a continuous representation for words and sentences, instead of using hand-engineered features as in other machine learning approaches. The seminar will introduce the main deep learning models used in natural language processing, allowing the attendees to gain hands-on understanding and implementation of them in Tensorflow.

Teaching staff

NameInstitutionCategoryDoctorTeaching profileAreaE-mail
AGIRRE BENGOA, ENEKOUniversity of the Basque CountryProfesorado PlenoDoctorBilingualComputer Languages and Systemse.agirre@ehu.eus
AZCUNE GALPARSORO, GORKAUniversity of the Basque CountryProfesorado AgregadoDoctorBilingualScience of Computation and Artificial Intelligencegorka.azcune@ehu.eus
BARRENA MADINABEITIA, ANDERUniversity of the Basque CountryProfesorado Adjunto (Ayudante Doctor/A)DoctorBilingualComputer Languages and Systemsander.barrena@ehu.eus
LOPEZ DE LACALLE LECUONA, OIERUniversity of the Basque CountryProfesorado Adjunto (Ayudante Doctor/A)DoctorBilingualComputer Languages and Systemsoier.lopezdelacalle@ehu.eus

Competencies

NameWeight
Learn skills to deal with strategies and tools for natural language processing.20.0 %
Learn skills to deal with machine learning methods for natural language processing.20.0 %
Ability to manage, adapt and improve the most relevant empirical methods for research in language technologies.20.0 %
Ability to handle multimodal representations.20.0 %
Ability to improve language understanding with visual information.20.0 %

Study types

TypeFace-to-face hoursNon face-to-face hoursTotal hours
Lecture-based203050
Applied computer-based groups4060100

Training activities

NameHoursPercentage of classroom teaching
Computer work practice, laboratory, site visits, field trips, external visits100.040 %
Lectures50.040 %

Assessment systems

NameMinimum weightingMaximum weighting
Essay, Individual work and/or group work50.0 % 50.0 %
Works and projects50.0 % 50.0 %

Learning outcomes of the subject

Show understanding of deep learning systems, as well as the main architectures used in NLP

Development of basic deep learning systems applied to NLP problems.

Show knowledge about the latest advances in deep learning for NLP

Ordinary call: orientations and renunciation

Sistema de Evaluación Continua

Herramientas y porcentajes de calificación:

Prueba escrita a desarrollar (%):

Realización de prácticas (ejercicios, casos o problemas) (%): 50

Trabajos individuales (%): 50



Sistema de Evaluación Final

Herramientas y porcentajes de calificación:

Prueba escrita a desarrollar (%): 50

Trabajos individuales (%): 50

Extraordinary call: orientations and renunciation

Sistema de Evaluación Final

Herramientas y porcentajes de calificación:

Prueba escrita a desarrollar (%): 50

Trabajos individuales (%): 50

Temary

1. Introduction to machine learning and NLP with Tensorflow

2. Multilayer Perceptron

3. Word embeddings and recurrent neural networks

4. Seq2seq, neural machine translation and better RNNs

5. Attention, Transformers and Natural Language Inference

6. Pre-trained transformers, BERTology

7. Bridging the gap between natural languages and the visual world

Bibliography

Basic bibliography

Yoav Goldberg's Primer. http://u.cs.biu.ac.il/~yogo/nnlp.pdf

Kyunghyun Cho¿'s course notes. http://arxiv.org/pdf/1511.07916.pdf

The online version of the Goodfellow, Bengio, and Courville Deep Learning textbook. http://www.deeplearningbook.org/