HPC MPI Exercise 1: Generating a Hosts File for MPI

Precondition

To complete this part of the exercise, you must be able to launch processes on a Gold or Maroon lab workstation without having to enter a password. If you cannot do so, you need to complete The One-Time Setup for rsh before proceeding.

MPI Hosts

MPI only works well on computers running Unix-family operating systems (e.g., Linux, MacOS), not Windows machines. In light of that, your next task is to create your own personal file of randomly-ordered Linux-workstation names. Later, we will tell MPI to use the hosts in this file to launch your processes, so if the order of the hosts in everyone's file is randomized, that should spread the processes out "evenly" among the lab machines.

Creating an MPI Hosts File

To make your life easier, Chris Wieringa has created the file /home/cs/374/genHosts.pl, which contains a Perl script that will generate a randomly-ordered list of the CS lab machines that are booted into Linux.

You will need to run this script frequently, so copy this file somewhere that you can find it easily. Start by making a new directory for this course (e.g., 374) in your home directory: 

   mkdir 374
and then use the cd command to change your working directory to that directory: 
   cd 374
From there, the command: 
   cp /home/cs/374/genHosts.pl .
will save a copy of Chris's script-file to your working directory.

The script-file is just a text file, and text files are not executable by default. You can verify this by entering: 

   ls -l
and noting that file is not executable.

To run it, you will need to change the file's permissions to allow execution: 

   chmod +x genHosts.pl

Then enter 

   ls -l
again to verify that the file's execute-permissions have changed.

Once your script-file is executable, your can run it: 

   ./genHosts.pl
and a random list of lab-machine names will appear on the screen. Since we want to store these machine names in a file, a better approach is to run the script and redirect its output to a file (e.g., hosts): 
   ./genHosts.pl > hosts
This will generate a randomized list of all the CS lab workstations that are currently booted and running Linux and store that list in the file named hosts.

Later in this exercise, MPI will launch processes on remote workstations in the order they are listed in your hosts file. Chris's script is supposed to ensure that the workstation you are using appears first in the list it generates so use the cat or less command to verify that that is the case.

Note that any time another student enters one of the labs and boots a workstation into Windows, or leaves one of the labs and shuts their workstation down, that workstation will no longer be running Linux, so your hosts file will be obsolete. Any time that happens, you will need to re-run your script to generate a new hosts file. This is both annoying and inconvenient; it is a key drawback to using the lab machines as a multiprocessor. Because of that, we will use the lab primarily for developing and debugging your MPI programs; to actually run your programs and measure their performance, we will use Calvin's cluster, where this is not an issue.

Once you can launch remote processes on the ulab machines without entering a password and have your own "randomized" personal machines file, your are ready to complete today's hands-on lab exercise.

CS > 374 > Exercises > 01 > Creating a Hosts File

This page maintained by Joel Adams.