Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI-DRIHM:Collaboration

From EGIWiki
Revision as of 18:34, 20 July 2013 by Sipos (talk | contribs)
Jump to navigation Jump to search
Welcome to the page of the EGI-DRIHM collaboration

EGI DRIHAM collaboration menu: Home Members Infrastructure Roadmap



Goal

The goal of the collaboration is to setup a web based science gateway for the hydrometeorogycal community to enable them running pre-defined simulation workflows useing resources from the European Grid Infrastructure. To achieve this goal the collaboration has to integrate varioushydrometeorological models with EGI, setup of a science gateway, develop workflows in the science gateway, develop unique interfaces for the workflows to simpify their use by scientists.

Details - the approach:

  • Demonstre the system at the 2nd DRIHM EC project review (Date: End of October)
  • Demo goal 1: demonstrate that hydrometeorology models can run on EGI
  • Demo goal 2: demonstrate that data transfer between models works in workflow/pipleline
  • Demo goal 3: demonstrate the workflows can be hidden behind unique interfaces, simplifying access for scientists
  • Workflow 1: WRF-NMM --> RainFARM --> DRIFT
  • Workflow 2: WRF-NMM --> RainFARM --> RIBS
  • Task 1: Integration of four models with EGI,
  • Task 2: setup of a science gateway and integrate with the ‘DRIHM Repository’
  • Task 3: development of workflows
  • Task 4: development of unique interfaces for the workflows

Background information: DRIHM project

  • DRIHM aims for facilitating the access to hydrometeorological data and models, the collaboration between meteorologists, hydrologists, and Earth science experts for accelerated scientific advances in hydrometeorological research (HMR).
  • Leader: Antonio Parodi (antonio.parodi @ cimafoundation.org)

EGI-DRIHM collaboration

  • Coordinator: Gergely Sipos, Technical Outreach Manager, EGI.eu (gergely.sipos@egi.eu)
  • Start Date: 17. May 2013
  • End Date: First phase to end by the 2nd DRIHM EC review (end of October)
  • Mailinglist: egi-drihm @ mailman.egi.eu
  • Meetings: Indico page (slides and minutes from meetings)

Infrastructure

  • The collaboration setup a new production VO on EGI: vo.drihm.eu.
    • VO ID Card: https://operations-portal.egi.eu/vo/update/283
    • vousers at drihm.eu: Write to all VO users
    • vousersupport at drihm.eu: Write to the VO support
    • vosecurity at drihm.eu: Write to security representatives of the VO
    • vomanagers at drihm.eu: Write to VO managers
  • In the first phase of the collaboration only gLite CREAM sites of the VO will be used in the first phase. Jobs will be submitted through the WMS.
  • In a later phase the GRAM5 sites of the VO can be also used through WMS. This requires the debugging of access to GRAM5 sites through the WMS (currently job submission through the WMS works to the Nikhef site, but not to the LRZ cluster). Daniele can work on this with the GRAM5 site admins and with WMS developers.
  • In a later phase the LRZ supercomputer site from PRACE SuperMuc) can be used to run the WRF model. This requires the installation of GRAM5 onto SuperMuc, and the acceptance of a community account/robot certificate on the site. (To avoid that individual users will have to have certificates and allocations on the site). Matteo to work on this and report back to the email list.
  • If the SuperMuc site becomes available for WRF, then the tools produced by the EGI-EUDAT-PRACE Pilots can be used in the workflows to transfer data between PRACE and EGI sites. (More info about the pilots: https://wiki.egi.eu/wiki/EGI_EUDAT_PRACE_collaboration.

Task 1

WRF integration

  • WRF is a parallel code, can run on HPC sites.
  • WRF requirements for the sites:
    • MPICH or MPICH2 or OPENMPI
    • 160-240 processors, but the simulations can be designed also to run on 50-100 processors
    • about 24 hours of physical simulation time, to an elapsed time of a single run of 144--96 hours (6--4 days).
    • Temporary disk space (GB): 80 GB for input, output, which corresponds to 24 h physical time, saving the data every 30 minutes of physical time. 28 GB for restart files, saving files every 6 hours of physical time.
  • Currently there are only two sites in the DRIHM VO where WRF successfully runs (on 40 CPUs), report by Daniele C. (The two sites are UNINA_EGEE and prague_cesnet_lcg2) Daniele to follow up the problems found with the other sites directly with the site managers. Report back to the list when additional sites become available for WRF.

RainFARM, DRIFT, RIBS integration

  • RainFARM, DRIFT, RIBS are sequential codes, have to run in a parameter study style on HTC grid sites
  • Hardware requirements have to be collected and added to the Wiki (action on Antonio and Fabio, who invite Luis)

Task 2: setup of a science gateway and integrate with the 'DRIHM Repository'

  • A gUSE 3.5.6 science gateway is already setup. URL has to be added to the Wiki page, so those who work on model integration and workflow development can get access.


VOMS

Primary/master instance

  • Hosted in Italy by Italian NGI at CNAF (in Bologna)
  • Host name: vomsmania.cnaf.infn.it
  • HTTPS port: 8443
  • VOMSES port: 15005

Secondary/slave instance

  • Hosted in Italy by Italian NGI in Naples
  • Host name: vomsIGI-NA.unina.it
  • HTTPS port: 8443
  • VOMSES port: 15005

Progress