Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI-InSPIRE:MAPPER-PRACE-EGI Task Force (MTF)

From EGIWiki
Jump to navigation Jump to search

Introduction to MTF

This EGI Wiki Page constitutes a working document through which MAPPER, PRACE and EGI InSPIRE projects will coordinate the activities of a jointly formed MAPPER-PRACE-EGI Task Force (MTF). The page will be administered on a day to day basis by EGI - Richard McLennan. All MTF members have a responsibility to contribute to the information recorded in these pages so as to help compile as full a picture as possible.

The MAPPER project (Multiscale APPlications on EuRopean e-infrastructures) aims to deploy a computational science environment for distributed multiscale computing, on and across European e-infrastructures. In order to further the project’s aim, MAPPER has initiated a collaboration with PRACE (PaRtnership for Advanced Computing in Europe) and with EGI InSPIRE (Integrated Sustainable Pan-European Infrastructure for Researchers in Europe). During an initial meeting between the 3 project teams held in May 2011 (Minutes of meeting), it was determined that as a first step, 2 applications should be integrated over the forthcoming 6 months to perform distributed multiscale computing and thus be used to promote the projects; the selected applications cover in 'stent restenosis' and 'nano material science'. This represents an important stage as MAPPER moves forward towards becoming part of the sustainable infrastructure used by the scientific communities at large. The coordination of activities relevant to this collaboration will be performed via an expressly formed Task Force comprising specialists from each of the 3 organisations.

Aims

  • MAPPER Project: At a practical level, the MAPPER Project Aim is for the MTF to prove 2 distributed multiscale computing applications using PrACE and the EGI (the aim is not to deliver sustainable ‘services’) to ??which customers?? by 24 Nov 2011.
  • EGI, PRACE Projects: There is an implicitly desired strategic effect of this work which is to further ‘Promote’ the ‘sustainable services’ that are collectively delivered by the 2 collaborating projects.

Timeframe

The MTF has been resourced for 6 months, ie the MTF should conclude its work by the end of November. Key dates are set out beneath:


Date Activity Remarks
13 May 11 Initial meeting of partners MAPPER/PRACE/EGI
1 Jun 11 Formation of Task Force
21 June 11 Informal gathering at ISC11 in Hamburg MAPPER/PRACE - EGI connected via Skype
Agree desired outcome/effect – Activity Aims MAPPER/PRACE/EGI
Clarify all requirements - develop/agree Statement of Requirements
Report initial findings
4-6 Jul 11 ??? Meeting in Cracow What meeting?
Implementation
Evaluation phase starts
Report further recommendations
24/25 Nov MTF Project Stage achievements Review Formal assessment that objectives have been achieved.
30 Nov 11 Task Force concludes
Present MAPPER application requirements to PRACE & EGI InSPIRE
May 2012 potential PRACE User Forum Derek Groen (UCL-MAPPER) to represent.


Composition of MTF – Roles and Responsibilities:

The activity to be undertaken via the MTF will ultimately have a wide impact throughout the communities of computation and scientific research and as such, there will need to be a clear understanding of a number of organisational issues: who defines what is required, who provides the budget/resources, who authorises changes, who manages day to day work and who defines the standards to be met. Answers to these questions will emerge as planning progresses.

In the first instance, the MTF is to be made up from members of each of the 3 participating Projects and will also include expert representatives from the 2 selected Applications. During the course of the investigation the MTF will necessarily call on additional support from within each of the Projects but this should be merely for the provision of expert technical information and clarification. Details of MTF members and key personalities are listed beneath.

Last Name First Name email mail list Affiliation Project Role:
Project owner ?? ??? ??? ??? Project owner
Kurowski Chris krzysztof.kurowski@man.poznan.pl Yes PSNC MAPPER MTF Leader
Groen Derek d.groen@ucl.ac.uk Yes UCL MAPPER MTF - MAPPER/PRACE info exchange
Zasada Stefan stefan.zasada@ucl.ac.uk Yes LMU MAPPER MTF
Saverchenko Ilya Ilya.Saverchenko@lrz.de Yes LRZ/LMU MAPPER MTF
Suter James j.suter@ucl.ac.uk Yes LMU MAPPER MTF
Rycerz Katarzyna kzajac@agh.edu.pl Yes Cyfronet MAPPER MTF
Mamoński Mariusz mamonski@man.poznan.pl Yes PSNC MAPPER MTF
Borgdorff Joris J.Borgdorff@uva.nl Yes UvA MAPPER MTF
Brewer Steve steve.brewer@egi.eu Yes EGI.eu EGI-Inspire MTF & Chief Community Officer
McLennan Richard richard.mclennan@egi.eu Yes EGI.eu EGI-InSPIRE MTF - Coordination
Axner Lilit lilit@kth.se KTH PRACE
Berg Axel Axel@sara.nl SARA PRACE
Bona Casas Carles c.bonacasas@uva.nl UvA MAPPER
Carteni Gabriele gabriele.carteni@bsc.es BSC PRACE
Donners John john.donners@sara.nl SARA PRACE
Erbacci Giovanni g.erbacci@cineca.it Cineca PRACE
Fiameni Giuseppe g.fiameni@cineca.it Cineca PRACE
Groen Derek d.groen@ucl.ac.uk UCL MAPPER
Hoekstra Alfons a.g.hoekstra@uva.nl UvA MAPPER MAPPER Project Director
Kurowski Chris krzysztof.kurowski@man.poznan.pl PSNC MAPPER
Lorenz Eric E.Lorenz@uva.nl UvA MAPPER
Michielse Peter p.michielse@nwo.nl NWO PRACE
Simpson Alan a.simpson@epcc.ed.ac.uk EPCC PRACE
Sipos Gregely gergely.sipos@egi.eu EGI.eu EGI-Inspire Senior User Community

Support Officer


Outline Description of Work

A Task Force normally carries out an initial assessment of an existing situation, identifying relevant problem issues such that they can be prioritised and so that potential solutions and courses of action can be appropriately formulated. The initial MAPPER/PRACE/EGI InSPIRE inter-project meeting of 13 May 2011 identified, in broad terms, the area that is to be focussed on and the need for a MAPPER Task Force to take the work forwards. The area to focus on was stated to be the integration of 2 specific applications that will demonstrate ‘distributed multiscale computing’ through the coupling of ‘PRACE-type systems’ with an ‘EGI type machine’ and a ‘local’ machine. Four general areas of activity were defined – these can be considered parallel activities rather than sequential though there are likely to be dependencies:


  • Initial investigation and planning
  • Identification of Requirements and data exchange
  • Services and Technology Synchronisation
  • Mid-term activities
  • Dissemination, Outreach & Promotion
  • Other related activities


Investigation of what can be done and formulation of possible options.

Activity Team:

  • Lead: Chris Kurowski (MAPPER)
  • PRACE members: Axel Berg, Giuseppe Fiameni (Technology), Gabriele Carteni (Service deployment)
  • EGI members: Steve Brewer, Tiziana Ferrari, Gergely Sipos, Richard McLennan
  • Coordination & publishing via EGI Wiki – Richard McLennan

Tasks:

  • Agree desired outcome/effect.
    • By R McLennan (EGI.eu): This activity needs to be seen as a milestone in a project that ultimately delivers MAPPER to the community at large as production infrastructure. The current milestone seems to deliver no more than a 'proof of concept'. If it is not seen as any more than a proof of concept, then it may be that there is no need for a Task Force, but this means that there will be many steps to be re-done properly later in the project.
  • Agree roles and responsibilities
  • Specify & document list and dates of Deliverables and milestones
    • ??by date?? collect documents that must be signed by end-users to proceed with the remote access.

Comment by G Sipos (EGI) for MAPPER PM. End users do not have to sign any paper. Every Virtual Organisation has its own “Acceptable Use Policy”. The users accept this with a click when they join the Virtual Organisation. Once we know which Virtual Organisation will be used (Point 1 or 2 above) we will know what Acceptable Use Policy the users will have to accept.

  • Develop & agree activity/stage plan
  • Agree resources and level of effort
    • Budget
    • Level of Effort
    • Equipment & Systems:
      • identify a list of machines that will be used for the first review

Comment by G Sipos (EGI): This decision depends on the flavour of middleware that your software (the workflow manager) can interact with. EGI Machines run gLite, ARC or UNICORE middleware. Which one is your workflow manager compatible with? Once we know the possible middleware flavour(s) we can:

        • 1) Give you access to a Virtual Organisation that has machines with this flavour
        • 2) Setup a new Virtual Organisation to collect machines from the infrastructure with this middleware.
  • Identify “customers” – involve customers and users; confirm their requirements:
    • ?? by date ?? by who ?? Two complementary groups of end-users must be defined to deal with:
      • a) application tools - WP7/8
      • b) core infrastructure services - WP4/5/6
    • ???deadline??? set by who?? provide detailed descriptions of applications, tools and core services which will be used for MAPPER demonstrations.
      • on a head node: QosCosGrid-Computing (a new OGF BES compliant service offering Advance Reservation features, integration via OGF DRMAA + AR APIs with the underlying queuing systems.
      • on working nodes: MUSCLE library, two multi-scale application packages involved in the 1st MAPPER review
  • Produce and gain approval of Statement of Requirements (SOR) – approval to be gained from Project Directors
    • ??by date?? by who?? All application and middleware developers have to have valid X.509 certificates. (accepted at least on one production machine that will be used for demonstrations). Produce named list of relevant personnel.
  • Report initial findings; recommend courses of action; select option, detail risks, detail quality standards.
  • Identify target users within user communities
    • ??by date?? by who?? Collect relevant information for account creation. (DN, first/second name, institution, etc.)
      • enable the access to preproduction machines, e.g create needed accounts, provide access tools
      • Comment by G Sipos (EGI): Users do not have accounts on production machines. They join Virtual Organisations with Personal User Certificates. Certificates can be obtained from National CAs (www.igtf.net) The type of user interfaces that we can provide depends on several factors:
        • 1) What interfaces are provided by your software? (the workflow environment where the multiscale applications will be defined and executed)
        • 2) What type of persons should be served? The application developers (who define the multi-scale workflows) or end-users (who will run these workflows with their own input)?
        • 3) What type of middleware you will use from EGI? (See decision point above)
    • Training
    • Support
  • Execution:
    • Management of Change
      • Who authorises?
      • Impact on other systems/users
    • Configuration Control
    • Risk Management
    • Quality Management
    • Dissemination Plan – Promotion & Advertising
    • Training Plan
    • Support Plan
    • Measure of Success????


Identification of Requirements and data exchange:

Activity Team:

  • Lead: Derek Groen (driving this activity),
  • MAPPER members: Bastien Chopard (MAPPER WP7 –Applications – leader), Alfons Hoekstra
  • PRACE members: Giueseppe Fiameni, Giovanni Erbacci
  • EGI member: Steve Brewer

Tasks:

  • Specify & document list and dates of Deliverables and milestones
    • Derek Groen, you mentioned that you had some "Use cases" which will help clarify this section and the specific requirements
  • Questions for MAPPER PM from T Ferrari (EGI.eu):
    • Which standards are used for job submission?
    • Are the jobs to be submitted (in particular to the grid resources) MPI jobs?
    • What middleware stack is deployed in the grid sites involved in the trial?
    • What are the user authentication issues?
    • Is advance reservation needed in step 1? (it is not in the UMD roadmap so far)
  • Document application data exchange requirements and PRACE Tier 0 issues.
  • ….etc.


Services and Technology Synchronisation

Activity Team:

  • Lead: Chris Kurowski (MAPPER)
  • MAPPER members: Ilya Saverchenko, Stefan Zasada,
  • PRACE Members: Giuseppe Fiameni, Gabriele Carteni
  • EGI:  ??

Tasks:

  • Specify & document list and dates of Deliverables and milestones
  • Document details of Scheduling, Monitoring systems
  • ….etc


Mid-term activities

Activity Team:

  • Lead:  ???
  • MAPPER members: Eric Lorenz, Carles Bona Casas
  • PRACE members: Alan Simpson, Giuseppe Fiameni

Tasks:

  • Specify & document list and dates of Deliverables and milestones
  • Investigate requirement for Petascaling
  • Start preparation for selected application access to Tier 1 and Tier 0
  • Create prioritised list of single scale applications codes
  • Document relevant requirements – expand SOR as required
  • ???Event Title?? meeting in Cracow – 4-6 Jul 2011
    • PRACE members to join.
    • Selected application developers to attend


Dissemination, Outreach & Promotion

Activity Team:

  • Lead: EGI ????
  • MAPPER members:  ????
  • PRACE Members:  ?????

Tasks:

  • Specify & document list and dates of Deliverables and milestones
  • …..etc


Other related activities.

There are likely to be a number of other related activities that will emerge as work progresses. Some such activities are already clear to the MTF and need to be considered:

Activity Team:

  • Lead: Chris Kurowski
  • Members: all

Tasks:

  • Add MAPPER questions to future PRACE / EGI User Surveys
    • Identify breadth and depth of reqt for MAPPER proposed Distributed Multiscale Computing
  • Confirm details of PRACE User Forum (May 2012??)
    • Derek Groen from UCL to take initial lead


Statement of Requirements

  • Who owns the Project?
  • Chris Kurowski owns Activity (of integrating 2 applications)
  • Who are the suppliers MAPPER, PRACE, EGI.eu, Application developers (x2)
  • Who are the customers? (ie, who will judge our success)
  • Who are the users?
  • Full SOR to be included as separate section.


Activity Plan

To be developed and included as subsection/Annex. Will include Execution plan, evaluation phase and details of ‘closure’.

  • Execution – roll-out and change management
  • Evaluation phase – success criteria, procedures, metrics etc
  • Training
  • Support task – help desk etc
  • Activity closure