Difference between revisions of "EGI-DRIHM:Collaboration"

From EGIWiki
Jump to: navigation, search
(Migrate content to DRIHM ICT status page)
(Migrate content to DRIHM ICT status page)
Line 39: Line 39:
 
* If the SuperMuc site becomes available for WRF, then the tools produced by the EGI-EUDAT-PRACE Pilots can be used in the workflows to transfer data between PRACE and EGI sites. (More info about the pilots: https://wiki.egi.eu/wiki/EGI_EUDAT_PRACE_collaboration.  
 
* If the SuperMuc site becomes available for WRF, then the tools produced by the EGI-EUDAT-PRACE Pilots can be used in the workflows to transfer data between PRACE and EGI sites. (More info about the pilots: https://wiki.egi.eu/wiki/EGI_EUDAT_PRACE_collaboration.  
 
* Read more about the resources that participate in the DRIHM VO in the Testbed section: [[EGI-DRIHM:Testbed]]
 
* Read more about the resources that participate in the DRIHM VO in the Testbed section: [[EGI-DRIHM:Testbed]]
 
== Task 1 ==
 
=== WRF integration ===
 
* WRF is a parallel code, can run on HPC sites.
 
* WRF requirements for the sites:
 
** MPICH or  MPICH2 or OPENMPI
 
**160-240 processors, but the simulations can be designed also to run on 50-100 processors
 
**about 24 hours of physical simulation time, to an elapsed time of a single run of 144--96 hours (6--4 days).
 
**Temporary disk space (GB): 80 GB for input, output, which corresponds to 24 h physical time, saving the data every 30 minutes of physical time. 28 GB for restart files, saving files every 6 hours of physical time.
 
* Currently there are only two sites in the DRIHM VO where WRF successfully runs (on 40 CPUs), report by Daniele C. (The two sites are UNINA_EGEE and prague_cesnet_lcg2) Daniele to follow up the problems found with the other sites directly with the site managers. Report back to the list when additional sites become available for WRF.
 
 
=== RainFARM, DRIFT, RIBS integration ===
 
* RainFARM, DRIFT, RIBS are sequential codes, have to run in a parameter study style on HTC grid sites
 
* Hardware requirements have to be collected and added to the Wiki (action on Antonio and Fabio, who invite Luis)
 
 
== Task 2: setup of a science gateway and integrate with the 'DRIHM Repository' ==
 
* A gUSE 3.5.6 science gateway is already setup: [http://sci-bus.drihm.eu:8080/liferay-portal-6.1.0/ http://sci-bus.drihm.eu:8080/liferay-portal-6.1.0/]
 
* The "DRIHM Repository" stores up to date executables for the models and the grid sites participating in the VO has to download these prior to model execution. The binaries are downloaded during the job preface (based on rsync), which is triggered by the job description. Person responsible: Nils from LMU
 
* CVMFS solution may be used later for the Repository, but this is condition of availability of this service on the sites. Tiziana to discuss CVMFS installation with the NGIs, report back to the email list. (It’s very unlikely that CVFMS would be production level on the sites by the DRIHM review.)
 
 
== Task 3: development of workflows ==
 
* Try WRF execution through the gUSE portal as a ‘one job workflow’ (Christian, Daniele C.) Inform us when this is done.
 
* Try RainFARM, DRIFT, RIBS execution through the gUSE portal as a ‘one job workflow’ (Antonio, Fabio, Luis, Daniele C.) Inform us when this is done.
 
* After the models run successfully individually through gUSE, integrate these together into the two target workflows.
 
* gUSE documentations for workflow developers (gUSE Manual Blue Series): http://guse.hu/documentation/how-to
 
 
== Task 4: development of unique interfaces for the workflows ==
 
* Daniele Dagostino already works on the portlets. Consult with Daniele C, Christian (WRF), Antonio, Fabio and Luis (RainFARM, DRIFT, RIBS) about what interfaces their models expect.
 

Revision as of 12:48, 16 January 2014

Welcome to the page of the EGI-DRIHM collaboration

EGI DRIHAM collaboration menu: Home Members Infrastructure Roadmap



Administrative

  • Coordinator: Gergely Sipos, Technical Outreach Manager, EGI.eu (gergely.sipos@egi.eu)
  • Start Date: 17. May 2013
  • End Date: First phase to end by the 2nd DRIHM EC review (end of October)
  • Mailinglist: egi-drihm @ mailman.egi.eu
  • Meetings: Indico page (slides and minutes from meetings)
  • Status of ongoing work is described atDRIHM ICT Status page

DRIHM project

  • DRIHM aims for facilitating the access to hydrometeorological data and models, the collaboration between meteorologists, hydrologists, and Earth science experts for accelerated scientific advances in hydrometeorological research (HMR).
  • Leader: Antonio Parodi (antonio.parodi @ cimafoundation.org)

Infrastructure

  • The collaboration setup a new production VO on EGI: vo.drihm.eu.
  • In the first phase of the collaboration only gLite CREAM sites of the VO will be used in the first phase. Jobs will be submitted through the WMS.
  • In a later phase the GRAM5 sites of the VO can be also used through WMS. This requires the debugging of access to GRAM5 sites through the WMS (currently job submission through the WMS works to the Nikhef site, but not to the LRZ cluster). Daniele can work on this with the GRAM5 site admins and with WMS developers.
  • In a later phase the LRZ supercomputer site from PRACE SuperMuc) can be used to run the WRF model. This requires the installation of GRAM5 onto SuperMuc, and the acceptance of a community account/robot certificate on the site. (To avoid that individual users will have to have certificates and allocations on the site). Matteo to work on this and report back to the email list.
  • If the SuperMuc site becomes available for WRF, then the tools produced by the EGI-EUDAT-PRACE Pilots can be used in the workflows to transfer data between PRACE and EGI sites. (More info about the pilots: https://wiki.egi.eu/wiki/EGI_EUDAT_PRACE_collaboration.
  • Read more about the resources that participate in the DRIHM VO in the Testbed section: EGI-DRIHM:Testbed