Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI-InSPIRE:WP6 Services for the Heavy User Community

From EGIWiki
Revision as of 13:49, 2 June 2010 by Jshiers (talk | contribs)
Jump to navigation Jump to search

Wiki page for EGI InSPIRE SA3 activities

WP6-SA3

This activity provides continued support for activities previously supported by EGEE while they transition to a sustainable support model within their own community or within the production infrastructure by:

  • Supporting the tools, services and capabilities required by different heavy user communities (HUCs)
  • Identifying the tools, services and capabilities currently used by the HUCs that can benefit all user communities and to promote their adoption
  • Migrating the tools, services and capabilities that could benefit all user communities into a sustainable support model as part of the core EGI infrastructure
  • Establishing e a sustainable support model for the tools, services and capabilities that will remain relevant to single HUCs

Task Leaders

To contact all task leaders (see below), send mail to inspire-sa3@mailman.egi.eu.

Involved partners (contacts / managers)

Germany - KIT-G (Horst Schwichtenberg, [horst.schwichtenberg@scai.fraunhofer.de])
Spain - CSIC (Isabel Campos Plasencia [isabel@campos-it.es])
Finland - CSC (Pekka Lehtovuori [Pekka.Lehtovuori@csc.fi])
France - CNRS (Johan Montagnat [johan@i3s.unice.fr])
Italy - INFN (Laura Perini [laura.perini@mi.infn.it])
Ireland - TCD (Brian Coghlan [coghlan@cs.tcd.ie], John Walsh [John.Walsh@cs.tcd.ie])
Poland - CYFRONET (Alex Kusznir [a.kusznir@cyfronet.pl])
Slovenia - IJS (Andrej Filipcic [andrej.filipcic@ijs.si])
Slovakia - UI SAV (Ladislav Hluchy [ladislav.hluchy@savba.sk])
UK - EMBL (Peter Rice [pmr@ebi.ac.uk])
CERN (Jamie Shiers [Jamie.Shiers@cern.ch], Maria Girone [Maria.Girone@cern.ch])

Send e-mail to inspire-sa3-partners@mailman.egi.eu

TSA3.1 Activity Management (3yr) (Jamie Shiers, CERN, [Jamie.Shiers@cern.ch])

The SA3 management is the responsibility of the Activity Manager, who reports to the UCO for the running of this activity and coordinates the work across the different tasks within the activity to meet the stated objectives. This activity will be represented in the USAG by the Activity Manager and the task leaders for the sub tasks in TSA3.2 to ensure that the services being offered to the general EGI community are meeting their needs. The domain specific activities will have their community specific mechanisms for supervision. The Activity Manager will pay particular attention to the provision of the shared services and tools task (TSA3.2) to ensure that all requests from all communities are correctly evaluated and prioritised into the work plan even if the work is being undertaken by a single community. All tasks will establish outreach and sustainability plans, and mechanisms for monitoring and gathering feedback on the delivery of their services. The work in SA3 has many benefits to other HUCs (e.g. EIRO and ESFRI) and more broadly the general DCI community. Some of the potential results from this activity include: dashboards customised to specific VOs, workflows and schedulers bridging different DCIs (e.g. DEISA, PRACE, EGI), support of MPI, frameworks for managing collections of jobs on DCIs, services for accessing relational data resources, secure data storage, visualisations tools, etc. Technical dissemination material will be generated by SA3 to inform all communities of this work.

TSA3.2 Shared Services and Tools (3yr) (Jamie Shiers, CERN, [Jamie.Shiers@cern.ch])

The services and tools within this task are already being used by multiple communities, and have been identified as having benefit to other specific communities or the general user community. It is expected that by the end of the project many of these services and tools will have been migrated into UMD (i.e. be supported independently of the EGI project by an external software provider) and deployed within EGI as part of the core, supported infrastructure. This migration will be justified by the continued adoption and use of these tools and services by HUCs and the broader EGI user community.

Subtasks:

• TSA3.2.1 Dashboards: Dashboards provide a generic framework to monitor sites and their services within a VO using tests specific to that community. Dashboards have emerged from within the HEP community, and are now being adopted by the LS community, to monitor their resources. (CERN, CNRS)

• TSA3.2.2 Applications: The GANGA and DIANE tools are both part of the EGEE RESPECT programme which recognises software that builds on top of the gLite platform. Although initially developed for the HEP community these tools have now gained traction in other communities, as they provide simple environments to manage large collections of tasks. The requirements of these tools will be integrated into the workplan. (CERN, UI SAV, ARNES)

Contacts: ARNES (Andrej Filipcic [andrej.filipcic@ijs.si])

• TSA3.2.3 Services: HYDRA and GReIC are services that have emerged from a single community that show potential for adoption in others. HYRDA allows an encryption key to be securely stored on distributed servers in order that storage elements can be used to store confidential data which is critical for the medical community securely. The GReIC service provides uniform relational and non-relational access to heterogeneous data sources and is currently being used to support bioinformatics and Earth Observation Systems. (CNRS, INFN)

Contacts: Giovanni Aloisio for GrelC (INFN)

• TSA3.2.4 Workflows and Schedulers: These tools are critical in integrating complex processes, generally involving multiple data sources and different computational resources, as needed within many disciplines. SOMA2 is a web-based workflow tool used for computational drug design and general molecular modelling. TAVERNA is used extensively by the bioinformatics community. The combination of the Kepler workflow engine and the Migrating Desktop platform are used by the Fusion community to run workflows requiring visualisation and interactive access on gLite and UNICORE-enabled resources. For simpler workflows and metascheduling scenarios the GridWay system is used by the Fusion community. Effort is provided to maintain the integration of these tools with the different systems. (CSIC, CSC, CYFRONET, EMBL)

Contacts: Francisco Castejon, francisco.castejon@ciemat.es Antonio Gomez, antonio.gomez@ciemat.es (CSIC), Tapani Kinnunen (tapani.kinnunen@csc.fi) (CSC)

• TSA3.2.5 MPI: Support for parallel computing (MPI) applications are critical for many user communities but the integration of this capability into the general infrastructure has been difficult. This task will focus on the improvement of the core services and software needed to support MPI, while engaging with two representative user communities (CCMST & Fusion) to ensure that the offered support meets their requirements. (CSIC, INFN, TCD)

Contacts: Enol Fernandez: enolfc@ifca.unican.es, John Walsh, TCD, School of Computer Science and Statistics, Trinity College Dublin, College Green, Dublin 2, Ireland, John.Walsh@cs.tcd.ie, Antonio Lagana' (CCMST) (INFN)

TSA3.3 Services for HEP (3yr) (Maria Girone, CERN, [Maria.Girone@cern.ch])

The HEP VO specific services are devoted to the support of the Grid interfaces of the 4 LHC experiments and are of particular importance now as we enter the exploitation phase of the world’s largest scientific machine – the Large Hadron Collider at CERN. They will ensure the LHC experiments can rely on the Grid for their data handling, as planned in the LCG Technical Design Report.

All LHC VOs require optimizations and improvements in a number of common areas: workload management, data management, monitoring, service deployment and operation. These optimisations – of which specific details are provided in the task description below – are an essential part of the move to sustainable operations, as well as handling additional load and complexity expecting from LHC data acquisition, (re-)processing and analysis. An agile structure will be used to organise this work allowing the team to react quickly to high-priority needs of the LHC VOs as experience of the accelerator grows. (CERN, INFN)

Contacts: CERN - Maria Girone [Maria.Girone@cern.ch], INFN - Laura Perini [laura.perini@mi.infn.it]

TSA3.4 Services for LS (3yr) (Johan Montagnat, CNRS, [johan@i3s.unice.fr])

This task will provide services and service deployment for the life sciences community. The resources available to the biomolecular community at the EBI will be integrated into the European production infratructure. Services identified as the highest priority for the life sciences activity technical coordination will be supported.

Contacts: Johan Montagnat, CNRS researcher (I3S laboratory), http://www.i3s.unice.fr/~johan, +33 (0) 492 96 51 03

TSA3.5 Services for A&A (3yr) (Claudio Vuerli [vuerli@oats.inaf.it], deputy Giuliano Taffoni, INAF, [giuliano.taffoni@iasfbo.inaf.it])

The A&A community will Grid enable visualisation and associated data interpretation tools (e.g. VisIVO) to show how data, that will in future be collected from projects such as the Sloan Disgital Sky Survey, the Low Frequency Array (LOFAR), the Square Kilometre Array (SKA) and others, will be analysed using the production infrastructure to rapidly provide visual representations of this data. HPC (e.g. DEISA/PRACE) resources and HTC resources will be integrated in this task as part of this proof of concept.

TSA3.6 Services for ES (3yr) (Horst Schwichtenberg, SCAI, [horst.schwichtenberg@scai.fraunhofer.de])

Implement, deploy and maintain the EGDR service to provide access from the grid to resources within GENESI-DR.

Milestones

HUC Contact points and the support model - CSC (month 1)

HUC Software Roadmap - INFN (month 4)

Services for High Energy Physics - CERN (month 4)

[ etc. ]

Deliverables

D6.1) Capabilities offered by the HUCs to other communities: This public report will illustrate how the functional capabilities being supported with this activity could be reused by other communities using European DCIs. Sufficient technical depth should be provided for potential adopters of DCI platforms to make an initial assessment of how they could work with the offered technologies. [month 4]

D6.2) Sustainability plans for the HUC activities: The HUC activities must plan for a sustainable future – either within their own community, as part of the generic production infrastructure, or through some other mechanism. The sustainability plan for each task within the activity will be defined in this public report by assessing the progress made to date and providing plans for the next year. [month 9]

D6.3) Annual Report on the HUC Tools and Services: A public report describing the current status of the tools and services provided within SA3, their activity over the preceding year, and the future activities including measures to assure sustainability of the current work. [month 11]

D6.4) Capabilities offered by the HUCs to other communities: This public report will illustrate how the functional capabilities being supported with this activity could be reused by other communities using European DCIs. Sufficient technical depth should be provided for potential adopters of DCI platforms to make an initial assessment of how they could work with the offered technologies. [month 16]

D6.5) Sustainability plans for the HUC activities: The HUC activities must plan for a sustainable future – either within their own community, as part of the generic production infrastructure, or through some other mechanism. The sustainability plan for each task within the activity will be defined in this public report by assessing the progress made to date and providing plans for the next year. [month 21]

D6.6) Annual Report on the HUC Tools and Services: A public report describing the current status of the tools and services provided within SA3, their activity over the preceding year, and the future activities including measures to assure sustainability of the current work. [month 23]

D6.7) Capabilities offered by the HUCs to other communities: This public report will illustrate how the functional capabilities being supported with this activity could be reused by other communities using European DCIs. Sufficient technical depth should be provided for potential adopters of DCI platforms to make an initial assessment of how they could work with the offered technologies. [month 28]

D6.8) Sustainability plans for the HUC activities: The HUC activities must plan for a sustainable future – either within their own community, as part of the generic production infrastructure, or through some other mechanism. The sustainability plan for each task within the activity will be defined in this public report by assessing the progress made to date and providing plans for the next year. [month 33]

D6.9) Annual Report on the HUC Tools and Services: A public report describing the current status of the tools and services provided within SA3, their activity over the preceding year, and the future activities including measures to assure sustainability of the current work. [month 35]