In previous years, the main techniques used to run applications in an efficient manner has been analyzed: Memory hierarchy, Pipelining, Superscalar and execution, multithreading, etc.

Furthermore, given that single-core performance is limited, a first approach to parallelism has also been covered, looking into multi-core and shared-memory architectures as well as vector and GPU computing.

This course covers how to further scale computing systems by looking into distributed memory systems, where massive parallelism is leveraged to reduce the execution time of complex problems through the utilization of thousands of processors or cores.

The first part of the course focuses on the architecture of parallel systems, with special emphasis on interconnection networks, message passing mechanisms and distributed coherency architectures.

The second part is devoted to parallel programming with MPI, the most common API to develop parallel applications.

In this part, we will learn how to parallelize an application, analyze its performance and generate the corresponding technical documentation, etc.

Learning Outcomes:

1. Understand the operation of distributed memory parallel systems.

2. Measure and analyze the performance of parallel computing systems and the applications running on them.

3. Examine different all the different aspects of interconnection networks.

4. Develop parallel applications with MPI.

In addition other general skills will be developed. (See the Computer Engineering degree website).

0. Review of parallel architecture concepts. Performance evaluation.

1. Interconnection networks and message passing.

2. Distributed memory coherency.

3. Parallel programming with MPI

The first part of the course (0-2) is delivered face-to-face, with two main types of activity: Theory lectures and exercise classes.

The second part (3) takes place in the lab where theoretical and practical work is interleaved, working in groups towards the parallelization of an application.

• Continuous Assessment System
• Final Assessment System
• Tools and qualification percentages:
  • The type of evaluation and percentages are specified in the next sections (%): 100

The evaluation systems are Continuous Evaluation and Global Evaluation.

The continuous evaluation system is the preferred one as per UPV/EHU regulations.



a. Continuous evaluation is the default system and only applies to the ordinary call. Active and steady participation is required: assistance to lectures, resolution of exercises and other homework and generally engaging with all activities: tests, labwork, presentations, etc.

Evaluation is as follows:

--1st part. Exercises + Exam: 4 marks

--2nd part. Exercises + Exam + Team MPI project: 6 marks



To pass the course, it is required to pass both parts with a minimum of 4/10 in the exams and project.

If academic malpractice is identified, the course will be failed, moving to the extraordinary call.



Students willing to opt out of the continuous evaluation must inform the teacher by means of an email before the last day of class.



b. Global evaluation in the established exam calendar will take place upon students opting out or being unable to continue the continuous evaluation.

Evaluation is as follows:

-- Exam (Full syllabus, including the MPI lab exam): 8 marks

-- Individual MPI project: 2 marks



To pass the course, it is required to obtain a minimum of 5/10 in the exam and project.

Not attending the exam is considered as renouncing to the evaluation and will be marked as "Not presented".

The extraordinary evaluation will follow the same procedure as the Global evaluation.

Materials for the course are available through eGela.

Basic bibliography

- Hennessy J.L., Patterson D.A.: Computer Architecture: A Quantitative Approach (5. ed.). Morgan Kaufmann, 2011.

- Culler D.E., Singh J.P.: Parallel Computer Architecture. A Hardware/Software Approach. Morgan Kaufmann, 1999.

- Duato J., Yalamanchili S., Ni L.: Interconnection Networks: An Engineering Approach. Morgan Kaufmann, 2003.

- Pacheco P.S.: Parallel Programming with MPI. Morgan Kaufmann, 1997.

- Groop W. et al: Using MPI. Portable Parallel Programming with the Message Passing Interface. The MIT Press, 1999.

- Snir M. et al.: MPI. The complete reference, Volume 1, The MPI core. The MIT Press, 1999

In-depth bibliography

- Dally W.J., Towles B.: Principles and Practices of Interconnection Networks. Morgan Kaufmann, 2004.
- Ma, S.; Huang, L.: Networks-on-chip. Morgan Kaufmann, 2015.
- Almeida F., Giménez D., Mantas J.M., Vidal A.M.: Introducción a la programación paralela. Thomson-Paraninfo, 2008.
- Quinn M.J.: Parallel Programming in C. McGraw-Hill, 2004.
- Barlas G.: Multicore and GPU programming. M. Kaufmann, 2015.

Journals

-- R. Espasa, M. Valero, J. Smith: Vector Architectures: Past, Present and Future. Int. Conf. on Supercomputing, 425-432, 1998.

-- Stenstrom P: A survey of cache coherence schemes for multiprocessors. Computer, v. 23, n. 6, 12-24, 1990.

-- Tomasevic M., Milutinovic V. (eds.). The Cache Coherence Problem in Shared Memory Multiprocessors: Hardware Solutions. IEEE Computer Society Press, 1993.

-- Mellor-Crummey J.M., Scott M.L. Algorithms for scalable synchronization on shared memory multiprocessors. ACM Transactions on Computer Systems, v. 9, n. 1, 21-65, 1991.

-- Adve S.V., Gharachorloo K. Shared memory consistency models: a survey. Computer, v. 29, n. 12, 66-76, 1996.

-- Hill M.D. Multiprocessors should support simply memory-consistency models. Computer, v. 31, n. 8, 28-34, 1998.

-- Wolfe M. High Performance Compilers for Parallel Computing. Addison Wesley, 1996.

-- Banerjee U., Eigenmann R., Nicolau A., Padua D. Automatic program parallelization. Proceedings of the IEEE, v. 81, n. 2, 211-243, 1993.

-- Lilja D.J. Exploiting the parallelism available in loops. Computer, v. 27, n. 2, 13-26, 1994

Web addresses

>> Fastest supercomputers: www.top500.org
>> InfiniBand infrastructure: www.infinibandta.org
>> MPI: www.mpich.org, www.open-mpi.org, www.mpi-forum.org
>> UPC-Unified Parallel C(DSM):upc.lbl.gov, upc.gwu.edu, www.intrepid.com/upc.html
>> Cluster computing: www.ieeetcsc.org, www.beowulf.org, www.lcic.org, www.linuxhpc.org