HPC Hybrid Exercise: Hands-On Lab


Combining MPI and OpenMP

Last week, we saw that OpenMP can be used for shared-memory parallel computing. In previous weeks, we used MPI for distributed-memory parallel computing. This week, we will see how to combine MPI and OpenMP for parallel computing on hybrid systems -- distributed-memory systems whose nodes are shared-memory multiprocessors. Virtually all modern HPC systems are hybrid systems.

Part I: Getting Started

To get started, download and run spmd and spmd2. Read the comment at the beginning of each source file to see what you should do with it.

Each example includes a Makefile to build it. Create a separate folder for each program and its Makefile, and take a moment to view the Makefile. When combining MPI and OpenMP within the same program, we:

These are both needed: mpicc to correctly process and link the MPI directives, and -fopenmp to link in the OpenMP library.

As usual, take the time to run these programs with varying numbers of processes and/or threads,, and trace their execution to see how they generate their outputs. Once you have explored these examples and are confident you understand how they work, you may continue to the next part of today's exercise.

Part II: Three Hybrid Puzzles

In class, we discussed three different patterns for coordinating the communication between the processes in a hybrid computation. In this section, we will explore 3 "puzzles" that try to model those coodination patterns.

Download the 3 puzzles -- puzzle, puzzle2, and puzzle3 -- but don't build or run them yet. Instead, find a neighbor in the lab to work with for the rest of today's session.

Puzzle 1. Using your favorite editor (or just cat), view the contents of puzzle.c. Trace its execution and try to predict what will be displayed when it is run as follows:

   mpirun -np 3 ./puzzle
Write down your prediction on a piece of paper. Then talk to your neighbor and compare predictions. If your predictions differ, try to resolve your disagreement and reach a consensus as to what will be displayed.

When you have reached consensus, build and run the program. Was your prediction correct or incorrect? If it was correct, congratulations!

If it was incorrect, work with your neighbor to figure out where your logic went astry. Trace through the source code to figure out how the output is being produced.

When you are confident you understand how puzzle works, continue on to the next puzzle.

Puzzle 2. As with the first puzzle, start by viewing the contents of puzzle2.c. Trace its execution and try to predict what will be displayed when it is run as follows:

   mpirun -np 3 ./puzzle2
As before, write down your prediction on a piece of paper. Then talk to your neighbor again and compare predictions. If your predictions differ, try to reach a consensus as to what will be displayed.

When you have reached consensus, build and run the program. Was your prediction correct or incorrect? If it was correct, congratulations!

If it was incorrect, work with your neighbor to figure out where you went wrong. Using the source code, work together to figure out how the output is being produced.

When you are confident you understand how puzzle2 works, continue on to the next puzzle.

Puzzle 3. View the contents of puzzle3.c. Trace its execution and try to predict what will be displayed when it is run as follows:

   mpirun -np 3 ./puzzle3
As before, write down your prediction on a piece of paper. Once more, compare predictions with your neighbor. If your predictions differ, try to reach a consensus as to what will be displayed.

When you have reached consensus, build and run the program. Was your prediction correct or incorrect? If it was correct, congratulations!

If it was incorrect, work with your neighbor to figure out where you went wrong. Using the source code, work together to figure out how the output is being produced.

Once you have figured that out, predict what puzzle3 will output when the #pragma omp master line is uncommented. Discuss with your neighbor until you have reached consensus. Then uncomment that line, rebuild and rerun puzzle3.

Was your prediction correct or incorrect? If it was correct, congratulations, you're done!

If it was incorrect, work with your neighbor to figure out where you went wrong. Work together to trace through the source code and figure out how the output is being produced.

When you are finished, you may continue to this week's project.


CS > 374 > Exercise > 08 > Hands-On Lab


This page maintained by Joel Adams.