MPI User Guide

MPI User Guide

This document is intended to help EGI user community to execute MPI applications on the Infrastructure.

Application Execution

MPI-start is the recommended way of starting MPI jobs in the infrastructure.

Site Support

Execution of MPI applications requires sites that properly support the submission and execution of parallel applications and the availability of a MPI implementation. Site administrators should check the MPI-Start Installation and Configuration manual with the relevant information about the configuration of sites. Since not all of them have this support enabled, special tags are published via the information system to allow users to discover which are the sites that can be used for their executions. Sites may also install different implementations (or flavours) of MPI. It is important therefore that users can use the information system to locate sites with the software they require.

Discovery

Discovery of resources is the first step that needs to be accomplished before the execution of applications. This can be done by using the GlueHostApplicationSoftwareRunTimeEnvironment attribute, which should include all the relevant MPI support information that allow users to locate the sites with the adequate software environment. The following sections describe the tags that site may publish

MPI-Start support

The recommended way of starting MPI applications in the EGI Infrastructure is using MPI-Start. Sites including support for MPI-Start must include in their GlueHostApplicationSoftwareRunTimeEnvironment attribute the tag MPI-START

MPI Implementation support

For each MPI implemenation supported, sites must publish a variable with the name of the MPI flavour that has been installed and tested. The supported flavours are: MPICH for MPICH, MPICH2 for MPICH2 , LAM for LA-MPI and OPENMPI for Open MPI. Most commonly supported flavours are Open MPI and MPICH2

Example:

• GlueHostApplicationSoftwareRunTimeEnvironment: MPICH2
• GlueHostApplicationSoftwareRunTimeEnvironment: OPENMPI

More specific version and compiler information can be also defined by the sites using variables with the form:

<MPI flavour>-<MPI version> or <MPI flavour>-<MPI version>-<Compiler>

These are not mandatory, although they should be published to allow users with special requirements to locate specific versions of MPI software. Users should assume gcc compiler suite is used if no other value is specified.

Examples:

• GlueHostApplicationSoftwareRunTimeEnvironment: OPENMPI-1.4.2
• GlueHostApplicationSoftwareRunTimeEnvironment: MPICH-1.2.7
• GlueHostApplicationSoftwareRunTimeEnvironment: OPENMPI-1.3.7-ICC

Network interconnects

Sites may publish the network interconnect available for the execution of MPI applications with a variable with the form:

MPI-<interconnect>

Currently the valid interconnects are: Ethernet, Infiniband, SCI, and Myrinet.

Examples

• GlueHostApplicationSoftwareRunTimeEnvironment: MPI-Infiniband

Shared homes

Sites supporting a shared filesystem for the execution of MPI applications publish the MPI_SHARED_HOME variable. If your application needs such feature, you should check the availability of that variable.

Sample Queries

If you are submitting your jobs through the gLite WMS, you should include in the Requirements expression the tags you want the site to support. The following example shows the requirements expression for a job that needs Open MPI and Infiniband and uses MPI-Start for execution:

Requirements  = member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment)
                && member("OPENMPI", other.GlueHostApplicationSoftwareRunTimeEnvironment)
                && member("MPI-INFINIBAND", other.GlueHostApplicationSoftwareRunTimeEnvironment);

The lcg-info command can be used to perform similar queries.

• Sites in ops.vo.ibergrid.eu VO that support MPICH2:

$ lcg-info --vo ops.vo.ibergrid.eu --list-ce --query 'Tag=MPICH2' 
- CE: ce02.ific.uv.es:8443/cream-pbs-infbandShort

• Sites in biomed VO that support Open MPI v1.4.4:

$ lcg-info --vo biomed --list-ce --query 'Tag=OPENMPI-1.4.4' 
- CE: ce01.kallisto.hellasgrid.gr:2119/jobmanager-pbs-biomed
- CE: cream01.kallisto.hellasgrid.gr:8443/cream-pbs-biomed
- CE: egeece01.ifca.es:2119/jobmanager-sge-biomed
- CE: egeece02.ifca.es:2119/jobmanager-sge-biomed
- CE: egeece03.ifca.es:2119/jobmanager-sge-biomed
- CE: gridce01.ifca.es:8443/cream-sge-biomed
- CE: gridce02.ifca.es:8443/cream-sge-biomed
- CE: ngiescream.i3m.upv.es:8443/cream-pbs-biomed

• Number of sites that support MPI in biomed VO:

$ lcg-info –vo biomed –list-ce –query 'Tag=*MPI*' –sed | wc
100

• Sites with Infiniband interconnect in biomed VO:

$ lcg-info --vo biomed --list-ce --query 'Tag=MPI-INFINIBAND' 
- CE: ce.reef.man.poznan.pl:2119/jobmanager-pbs-biomed
- CE: ce002.ipp.acad.bg:2119/jobmanager-pbs-biomed
- CE: cr1.ipp.acad.bg:8443/cream-pbs-biomed
- CE: creamce.reef.man.poznan.pl:8443/cream-pbs-biomed

• Sites with MPI-Start, Open MPI and Ethernet interconnect in biomed VO:

$ lcg-info --vo biomed --list-ce --query 'Tag=OPENMPI,Tag=MPI-START,Tag=MPI-ETHERNET' 
- CE: glite.univ.kiev.ua:2119/jobmanager-pbs-grid
- CE: glite.univ.kiev.ua:8443/cream-pbs-grid
- CE: grid-lab-ce.ii.edu.mk:2119/jobmanager-pbs-biomed