HPC MPI Exercise 2: Hands-On Lab

The Master-Worker Pattern

Begin by making a new directory for this exercise in your course directory. In that directory, save a copy of the C program masterWorker.c. This program is another parallel "hello world"-style MPI program, but this one demonstrates the Master-Worker pattern, another commonly used parallel programming pattern. Take a few minutes to look over the program.

How does masterWorker.c differ from spmd.c, which we looked at last time?

The basic structure of this pattern is:

   if (id == 0) {
      // put the master's code here
   } else {
      // put the worker's code here
   }

Of course, if the master and/or worker's code is more than a few lines, you should write a function to perform its task, to keep your code modular:

   if (id == 0) {
      runMaster();
   } else {
      runWorker();
   }

To compile this program, we will use the mpicc command again:

   mpicc masterWorker.c -Wall -o masterWorker

To run the program in the Unix Lab, we will again use the mpirun command. Start by running the program with a single process:

  mpirun -np 1 -machinefile hosts ./masterWorker

Then run it again using 2, 4, 6, and 8 processes and compare its behavior against its source code, until you understand how it is generating the behavior you observe.

The Message-Passing Pattern

When you are confident that you understand how masterWorker.c is working, you are ready to proceed to our second example program -- messagePassing.c -- which introduces the message-passing commands in MPI.

Take a few minutes to look it over, to try to figure out what it is doing. Then compile and run it with differing numbers of processes, as we have done with our previous programs. (Note that in order for the compiler to find the definition of the sqrt() function, you will need to tell mpicc to link in the math library. by adding the -lm switch to the end of your compile command.)

As this program illustrates, MPI's version of the message passing pattern is accomplished using paired send and receive commands. These particular commands are blocking, meaning:

If process A executes a send command to process B, but B has not executed a corresponding receive, A's execution may be blocked until B executes that receive.
If process B executes a receive command from process A, but A has not executed a corresponding send, B's execution is blocked until A executes that send.

The general form of MPI's send command is:

   MPI_Send( addressOfFirstValue,
             numberOfValues,
             typeOfValues,
             idOfDestination,
             messageTag,
             communicator );

Let's examine these arguments one at a time:

addressOfFirstValue. The first argument is the address of the first value being sent. As we shall see shortly, this command can be used to send multiple values (by storing those values in an array), so when we use it to send the value of a single variable (as does messagePassing.c), we must use the address-of operator (&).
numberOfValues. The second argument is the number of values being sent -- an integer value.
typeOfValues. The third argument is the type of the values being sent. MPI defines its own hardware-independent types for transmission of values across a network. (The sender of a value might be running on a big-endian machine while the receiver might be running on a little-endian machine.)
MPI's types correspond closely with C's primitive data types. A few of the more common ones are:
- MPI_CHAR, the C char type.
- MPI_INT, the C signed int type.
- MPI_UNSIGNED, the C unsigned type.
- MPI_FLOAT, the C float type.
- MPI_DOUBLE, the C double type.
Click here for a complete list of MPI's predefined types.
idOfDestination. The fourth argument is the rank or id of the destination process.
messageTag. The fifth argument is an integer used to tag the message. 0 is commonly used for untagged messages. In messagePassing.c, we tag the first message being sent with 1 and tag the second message being sent with 2.
communicator. The final argument is the communicator or group of processes used to assign process ranks. For most of the projects in this course, we will use the communicator named MPI_COMM_WORLD, which is one of the things that the MPI_Init() command sets up.

MPI's receive command, is roughly symmetric to its send command:

   MPI_Recv( addressOfReceiveBuffer,
             maxiumNumberOfValues,
             typeOfValues,
             idOfSender,
             messageTag,
             communicator,
             messageStatus );

Examining these one at a time:

addressOfReceiveBuffer. The first argument is the address where the first value received should be stored.
maximumNumberOfValues. The second argument is the capacity of the first argument. This should be less than or equal to the second argument of the corresponding send.
typeOfValues. The third argument is the type of value being received. This should exactly match the type of value being sent.
idOfSender. The fourth argument is id of the process from whom a message should be received (or MPI_ANY_SOURCE to receive a message from any sender).
messageTag. The fifth argument is the tag of the message to be received (or MPI_ANY_TAG to receive a message with any tag).
communicator. The six argument is the same as with the send command.
messageStatus. The final argument is the address of an MPI_Status struct, which contains two fields: MPI_SOURCE and MPI_TAG. These can be used to determine the sender or tag of the message received when MPI_ANY_SOURCE or MPI_ANY_TAG are used.

Message Passing with Arrays

As mentioned previously, MPI's send-receive mechanism allows us to send multiple values in a single message by storing those values in an array. To see how this differs from sending a single value, download and open arrayPassing.c, and compare the send and receive commands in it to those in messagePassing.c.

Since the first arguments to MPI_Send() and MPI_Recv() are addresses, and since the value associated with a C/C++ array's name is the address of the first element of that array, we do not need to use the address-of operator when that first argument is an array. The arrays in arrayPassing.c are both dynamically allocated, but this works the same for statically allocated C arrays.

Note that the 2nd argument of the send command should be the number of items in the array, to avoid sending unnecessary values. Since we are sending a C-string (which is terminated by a null (0) character), and since the strlen() function does not include that null character in the length it reports, we must add 1 to that length to send the string and the null character that terminates it. (This is only an issue when sending char arrays.)

By contrast, the 2nd argument to the receive command should be the maximum capacity (the number of elements) of the array, to avoid overflowing the array.

Note also that the program in arrayPassing.c uses char arrays (in C++, string is a class, and C has no classes). This same array-sending mechanism works with arrays of arbitrary types.

Compile arrayPassing.c; then run it with varying numbers of processes, to see how its behavior differs from messagePassing.c.

You will be using both the Master-Worker and Message-Passing patterns in this week's homework project; when you are confident you understand how these patterns work, feel free to proceed to that project.

CS > 374 > Exercise > 02 > Hands-On Lab

This page maintained by Joel Adams.