Argonne National Laboratory

Accelerator Operations & Physics

Advance Photon Source
A U.S. Department of Energy, Office of Science,
Office of Basic Energy Sciences national synchrotron x-ray research facility

US Department of Energy
Argonne Home > Advanced Photon Source > Accelerator Systems Division > Accelerator Operations & Physics

Prebuilt Pelegant usage for multi-core computers

Y. Wang, R. Soliday and M. Borland

MPI installation

MPI-2 standard is strongly encouraged to use in the literature. We choose one popular implementation, MPICH2, from Argonne National Laboratory to build binary Pelegant for distribution. All the other implementations of the MPI standard can also be used to build Pelegant from source code.

The official URL for MPICH2 is:
http://www.mcs.anl.gov/research/projects/mpich2/

For Windows users you can download and install the binary version of MPICH2.

For Linux and Mac users you will need to download the source code mpich2-{version}.tar.gz and run:
gzip -d mpich2-{version}.tar.gz
tar -xf mpich2-{version}.tar
cd mpich2-{version}
./configure -prefix={install_dir}
make
make install

Then add the {install_dir}/bin to your PATH. The script install_mpi.sh can be used to download and compile MPICH2.

Pelegant installation

Precompiled binary version of Pelegant are available for Windows, Linux and Mac operating systems. These can be downloaded from our Software page in the elegant distributions. It is also possible to compile Pelegant from source code but this is much more complicated.

System configuration

Modern Linux operating systems will have an issue with MPI programs that have more threads then CPU cores that will result in suboptimal speeds. To overcome this you will need to run the following as root:
echo 1 > /proc/sys/kernel/sched_compat_yield

You may also want to install a queuing system such as Grid Engine but it is not required.

Running Pelegant

The detailed information about running jobs with MPICH2 can be found in the "User's Guide" from MPICH2 homepage.
Here is a quick start (the run_Pelegant script can be used alternatively):

If this is your first time to use MPICH2, the following setup is required on Linux and Mac systems:
echo "MPD_SECRETWORD=secretword" > ~/.mpd.conf
chmod 600 ~/.mpd.conf

Windows Users:
From a command prompt or Cygwin shell:
mpiexec -n 4 Pelegant file.ele

Linux and Mac Users:
Now we can start mpd daemon to run Pelegant job:
mpdboot
mpiexec -host $HOSTNAME -n 4 Pelegant file.ele
mpdallexit

If your computer has less than 4 cores, please follow the procedure in the following step to run Pelegant efficiently with numberOfCore+1 processes. The master CPU, which is used for I/O operations and tracking serial element, requires very little CPU time. For machine with more than 4 cores, launching the same number of processes as the number of cores should achieve reasonable speedup.

Hopefully by following these steps the installation will not be too difficult. If you have problems please email me at soliday@aps.anl.gov