Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Parallel Computing Support User Guide"

From EGIWiki
Jump to navigation Jump to search
Line 29: Line 29:
== Using MPI-START and mpiexec to perform non mpi workloads ==
== Using MPI-START and mpiexec to perform non mpi workloads ==


In the example above, the hello_bin command will run with a single argument "hello" on each of the nodes allocated to the job.
Both OpenMPI and MPICH2 define a number of environment variables that are available to every MPI process. In particular, they export variables which relate to the number of process slots allocated to the job and to the MPI Rank of the processes. Using this information one can nominate a "master" or coordinator process in the set of processes. This allows us to accommodate some master/slave use-cases.  
 
<nowiki>
#!/bin/bash
if test x"$OMPI_COMM_WORLD_RANK" = x"0" ; then
### Code for coordinating master process
else
### Code for slave processes
fi
</nowiki>

Revision as of 15:16, 27 April 2012

Summary

This page discusses support for generic parallel computing jobs on the EGI infrastructure. We consider using the MPI-START framework as a means for launching multiple jobs on a cluster. The are several clearly apparent application areas:

* Hadoop-On-Demand/myHadoop
* Charm++
* Parallel R

This is a work in progress.

JDL requirements

As we are using the MPI-START framework, the format of the JDL is the same as for an MPI job. However, the executable hello_bin may launch any process

JobType       = "Normal";
CPUNumber     = 4;
Executable    = "starter.sh";
Arguments     = "OPENMPI hello_bin hello";
InputSandbox  = {"starter.sh", "hello_bin"};
OutputSandbox = {"std.out", "std.err"};
StdOutput     = "std.out";
StdError      = "std.err";
Requirements  = member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment) && member("OPENMPI", other.GlueHostApplicationSoftwareRunTimeEnvironment);



Using MPI-START and mpiexec to perform non mpi workloads

Both OpenMPI and MPICH2 define a number of environment variables that are available to every MPI process. In particular, they export variables which relate to the number of process slots allocated to the job and to the MPI Rank of the processes. Using this information one can nominate a "master" or coordinator process in the set of processes. This allows us to accommodate some master/slave use-cases.

#!/bin/bash if test x"$OMPI_COMM_WORLD_RANK" = x"0" ; then ### Code for coordinating master process else ### Code for slave processes fi