Applied Mathematics AM 117

- Frequently Asked Questions for the Parallel Programming Assignment -

For technical questions please send email to Xiaoliang Wan (xlwan@dam.brown.edu)

How do I log into the parallel machine?

To enter babe (the parallel machine), you must use secure shell (ssh). For example, to log in you can type the following:

> ssh -l [username] babe.cascv.brown.edu

Then you can check the status of each node in the cluster by typing:

> llstatus

'Busy' means all CPUs are running, 'Run'means some but not all CPUs are running.
Where do I compile?

You will compile on babe.cascv.brown.edu but you have to run your programs on some computing nodes, e.g., compute-0-0.sp.priv or compute-0-0.sp.priv. Some compilers are NOT set up properly on the computing nodes.
What kind of machine is this?

These 2 nodes (compute-0-0.sp.priv, compute-0-1.sp.priv) have 4 CPUs per node (total of 8) and 3 GBytes of memory per node (total of 6 GBytes). They are equipped with the Motorola PowerPC 604e processor and they are operating under the IBM version UNIX, called AIX. These nodes are the debug nodes. Do not expect to get good performace on your program timings.
How do I compile a serial program on these nodes?

> xlC -qarch=ppc -qtune=604 -o serial_mat_vect serial_mat_vect.c

You can also use the IBM FORTRAN compilers xlf, f77, xlf90, xlf95 or the C compiler xlc.
How do I compile a parallel program on these nodes?

> mpCC -qarch=ppc -qtune=604 -o parallel_mat_vect parallel_mat_vect.c

You can also use the IBM FORTRAN compilers mpxlf, mpxlf90 or the C compiler mpcc.
How do I run a parallel program on these nodes?

You have to invoke the command poe which stands for Parallel Operating Environment. First you have to create a "hostfile" (named, say, hosts) listing line by line the names of the nodes you will be using as many times as the number of processors you need. For example, if you were to run a 4-processor job on node compute-0-0.sp.priv, then your hostfile would list the string compute-0-0.sp.priv 4 times like this:
```
babe> cat hosts 
compute-0-0.sp.priv
compute-0-0.sp.priv
compute-0-0.sp.priv
compute-0-0.sp.priv
```
If you were to run a 6-processor job with 4 processors on on node compute-0-0.sp.priv and 2 processors on compute-0-1.sp.priv, then your hostfile would look like this:
```
babe> cat hosts 
compute-0-0.sp.priv
compute-0-0.sp.priv
compute-0-0.sp.priv
compute-0-0.sp.priv
compute-0-1.sp.priv
compute-0-1.sp.priv
```
Once the hostfile (called "hosts" in our example) is ready, if you want to run, eg, a 4-processor job you would type

> poe parallel_mat_vect -hfile hosts -procs 4
How can I get better timings for my program?

You can submit to the LoadLeveler (a daemon that handles requests for batch access) but if you would like to do this please email the teaching assistant for a sample submit script.
Where are the LAPACK libraries on babe.cascv.brown.edu?

You can find one copy of lapack at TA's home directory: ~xlwan/liblapack_ppc.a. You do NOT have to use it, but you are allowed to do so if you feel comfortable with this package. These routines just provide simple (and optimized) ways of performing linear algebra operations.
How do I compile/link a serial program which contains LAPACK routines?

Suppose that I want to compile and link my C++ HW6.cpp file using the lapack library at /users/my_account/. At the command prompt, type:

> xlC -o HW6 HW6.cpp -L/users/my_account/ -llapack_ppc -lessl -lm -lxlf90 -lxlf

This will produce the compiled and linked executable HW6.
What changes when I want to compile/link a parallel program?

Instead of using xlC, compile with mpCC.
What do I need to include within my C++ file to use LAPACK routines?

When using LAPACK routines, you must declare them within an extern "C" statement as shown below.
```
 extern "C"{
 void dgbsv(int*,int*,int*,int*,double*,int*,int*,double*,int*,int*);
 }
```
Notice that pointers are passed. This is because the LAPACK routines are written in FORTRAN, and hence require that addresses be passed to the routine (since FORTRAN passes variables by reference and not by value as C++ does).