View source for EGI-InSPIRE:WP6 Services for the Heavy User Community

Wiki page for EGI InSPIRE SA3 activities

=WP6-SA3=

This activity provides continued support for activities previously supported by EGEE while they transition to a sustainable support model within
their own community or within the production infrastructure by:
* Supporting the tools, services and capabilities required by different heavy user communities (HUCs)
* Identifying the tools, services and capabilities currently used by the HUCs that can benefit all user communities and to promote their adoption
* Migrating the tools, services and capabilities that could benefit all user communities into a sustainable support model as part of the core EGI infrastructure
* Establishing e a sustainable support model for the tools, services and capabilities that will remain relevant to single HUCs

==Task Leaders==

To contact all task leaders (see below), send mail to inspire-sa3@mailman.egi.eu. 

==Involved partners (contacts / managers)==

Germany - KIT-G (Horst Schwichtenberg)<br>
Spain   - CSIC (Isabel Campos Plasencia [isabel@campos-it.es])<br> 
Finland - CSC  (Pekka Lehtovuori [Pekka.Lehtovuori@csc.fi]) <br>
France  - CNRS (Johan Montagnat [johan@i3s.unice.fr]) <br> 
Italy   - INFN (Laura Perini [laura.perini@mi.infn.it])<br> 
Ireland - TCD (Brian Coghlan [coghlan@cs.tcd.ie], John Walsh [John.Walsh@cs.tcd.ie])<br>
Poland  - CYFRONET (6 PM - no contact)<br>
Slovenia - ARNES (3 PM - no contact) <br>
Slovakia - UI SAV (Ladislav Hluchy [ladislav.hluchy@savba.sk])<br>
UK - EMBL (Peter Rice [pmr@ebi.ac.uk])<br>
CERN (Jamie Shiers [Jamie.Shiers@cern.ch], Maria Girone [Maria.Girone@cern.ch])

==TSA3.1 Activity Management (3yr) (Jamie Shiers, CERN, [Jamie.Shiers@cern.ch])==

The SA3 management is the responsibility of the Activity Manager, who reports to the UCO for the running of
this activity and coordinates the work across the different tasks within the activity to meet the stated objectives.
This activity will be represented in the USAG by the Activity Manager and the task leaders for the sub tasks in
TSA3.2 to ensure that the services being offered to the general EGI community are meeting their needs. The
domain specific activities will have their community specific mechanisms for supervision. The Activity Manager
will pay particular attention to the provision of the shared services and tools task (TSA3.2) to ensure that all
requests from all communities are correctly evaluated and prioritised into the work plan even if the work is being
undertaken by a single community. All tasks will establish outreach and sustainability plans, and mechanisms for
monitoring and gathering feedback on the delivery of their services.
The work in SA3 has many benefits to other HUCs (e.g. EIRO and ESFRI) and more broadly the general DCI
community. Some of the potential results from this activity include: dashboards customised to specific VOs,
workflows and schedulers bridging different DCIs (e.g. DEISA, PRACE, EGI), support of MPI, frameworks for
managing collections of jobs on DCIs, services for accessing relational data resources, secure data storage,
visualisations tools, etc. Technical dissemination material will be generated by SA3 to inform all communities of
this work.

==TSA3.2 Shared Services and Tools (3yr) (Jamie Shiers, CERN, [Jamie.Shiers@cern.ch])==

The services and tools within this task are already being used by multiple communities, and have been identified
as having benefit to other specific communities or the general user community. It is expected that by the end of
the project many of these services and tools will have been migrated into UMD (i.e. be supported independently
of the EGI project by an external software provider) and deployed within EGI as part of the core, supported
infrastructure. This migration will be justified by the continued adoption and use of these tools and services by
HUCs and the broader EGI user community.

Subtasks:

• TSA3.2.1 Dashboards: Dashboards provide a generic framework to monitor sites and their services within a VO
using tests specific to that community. Dashboards have emerged from within the HEP community, and are now
being adopted by the LS community, to monitor their resources. (CERN, CNRS)

• TSA3.2.2 Applications: The GANGA and DIANE tools are both part of the EGEE RESPECT programme which
recognises software that builds on top of the gLite platform. Although initially developed for the HEP community
these tools have now gained traction in other communities, as they provide simple environments to manage
large collections of tasks. The requirements of these tools will be integrated into the workplan. (CERN, UI SAV, ARNES)

• TSA3.2.3 Services: HYDRA and GReIC are services that have emerged from a single community that show
potential for adoption in others. HYRDA allows an encryption key to be securely stored on distributed servers
in order that storage elements can be used to store confidential data which is critical for the medical community
securely. The GReIC service provides uniform relational and non-relational access to heterogeneous data
sources and is currently being used to support bioinformatics and Earth Observation Systems. (CNRS, INFN)

Contacts: Giovanni Aloisio for GrelC (INFN)    

• TSA3.2.4 Workflows and Schedulers: These tools are critical in integrating complex processes, generally
involving multiple data sources and different computational resources, as needed within many disciplines.
SOMA2 is a web-based workflow tool used for computational drug design and general molecular modelling.
TAVERNA is used extensively by the bioinformatics community. The combination of the Kepler workflow engine
and the Migrating Desktop platform are used by the Fusion community to run workflows requiring visualisation
and interactive access on gLite and UNICORE-enabled resources. For simpler workflows and metascheduling
scenarios the GridWay system is used by the Fusion community. Effort is provided to maintain the integration of
these tools with the different systems. (CSIC, CSC, CYFRONET, EMBL)

Contacts: Francisco Castejon, francisco.castejon@ciemat.es Antonio Gomez, antonio.gomez@ciemat.es (CSIC), Tapani Kinnunen (tapani.kinnunen@csc.fi) (CSC)

• TSA3.2.5 MPI: Support for parallel computing (MPI) applications are critical for many user communities but
the integration of this capability into the general infrastructure has been difficult. This task will focus on the
improvement of the core services and software needed to support MPI, while engaging with two representative
user communities (CCMST & Fusion) to ensure that the offered support meets their requirements. (CSIC, INFN, TCD)

Contacts: Enol Fernandez: enolfc@ifca.unican.es, John Walsh, TCD, School of Computer Science and Statistics, Trinity College Dublin, College Green, Dublin 2, Ireland, John.Walsh@cs.tcd.ie, Antonio Lagana' (CCMST) (INFN)

==TSA3.3 Services for HEP (3yr) (Maria Girone, CERN, [Maria.Girone@cern.ch])==

The HEP VO specific services are devoted to the support of the Grid interfaces of the 4 LHC experiments and
are of particular importance now as we enter the exploitation phase of the world’s largest scientific machine –
the Large Hadron Collider at CERN. They will ensure the LHC experiments can rely on the Grid for their data
handling, as planned in the LCG Technical Design Report.

All LHC VOs require optimizations and improvements in a number of common areas: workload management,
data management, monitoring, service deployment and operation. These optimisations – of which specific details
are provided in the task description below – are an essential part of the move to sustainable operations, as well
as handling additional load and complexity expecting from LHC data acquisition, (re-)processing and analysis.
An agile structure will be used to organise this work allowing the team to react quickly to high-priority needs of
the LHC VOs as experience of the accelerator grows. (CERN, INFN)

Contacts: CERN - Maria Girone [Maria.Girone@cern.ch], INFN - Laura Perini [laura.perini@mi.infn.it]

==TSA3.4 Services for LS (3yr) (Johan Montagnat, CNRS, [johan@i3s.unice.fr])==

This task will provide services and service deployment for the life sciences community. The resources available
to the biomolecular community at the EBI will be integrated into the European production infratructure. Services
identified as the highest priority for the life sciences activity technical coordination will be supported.

Contacts: Johan Montagnat, CNRS researcher (I3S laboratory), http://www.i3s.unice.fr/~johan, +33 (0) 492 96 51 03

==TSA3.5 Services for A&A (3yr) (Claudio Vuerli, INAF)==

The A&A community will Grid enable visualisation and associated data interpretation tools (e.g. VisIVO) to
show how data, that will in future be collected from projects such as the Sloan Disgital Sky Survey, the Low
Frequency Array (LOFAR), the Square Kilometre Array (SKA) and others, will be analysed using the production
infrastructure to rapidly provide visual representations of this data. HPC (e.g. DEISA/PRACE) resources and
HTC resources will be integrated in this task as part of this proof of concept.

==TSA3.6 Services for ES (3yr) (Horst Schwichtenberg, SCAI)==

Implement, deploy and maintain the EGDR service to provide access from the grid to resources within
GENESI-DR.

==Milestones==

HUC Contact points and the support model - CSC (month 1)

HUC Software Roadmap - INFN (month 4)

Services for High Energy Physics - CERN (month 4)

[ etc. ]

==Deliverables==

D6.1) Capabilities offered by the HUCs to other communities: This public report will illustrate how the functional
capabilities being supported with this activity could be reused by other communities using European DCIs.
Sufficient technical depth should be provided for potential adopters of DCI platforms to make an initial
assessment of how they could work with the offered technologies. [month 4]

D6.2) Sustainability plans for the HUC activities: The HUC activities must plan for a sustainable future – either
within their own community, as part of the generic production infrastructure, or through some other mechanism.
The sustainability plan for each task within the activity will be defined in this public report by assessing the
progress made to date and providing plans for the next year. [month 9]

D6.3) Annual Report on the HUC Tools and Services: A public report describing the current status of the tools
and services provided within SA3, their activity over the preceding year, and the future activities including
measures to assure sustainability of the current work. [month 11]

D6.4) Capabilities offered by the HUCs to other communities: This public report will illustrate how the functional
capabilities being supported with this activity could be reused by other communities using European DCIs.
Sufficient technical depth should be provided for potential adopters of DCI platforms to make an initial
assessment of how they could work with the offered technologies. [month 16]

D6.5) Sustainability plans for the HUC activities: The HUC activities must plan for a sustainable future – either
within their own community, as part of the generic production infrastructure, or through some other mechanism.
The sustainability plan for each task within the activity will be defined in this public report by assessing the
progress made to date and providing plans for the next year. [month 21]

D6.6) Annual Report on the HUC Tools and Services: A public report describing the current status of the tools
and services provided within SA3, their activity over the preceding year, and the future activities including
measures to assure sustainability of the current work. [month 23]

D6.7) Capabilities offered by the HUCs to other communities: This public report will illustrate how the functional
capabilities being supported with this activity could be reused by other communities using European DCIs.
Sufficient technical depth should be provided for potential adopters of DCI platforms to make an initial
assessment of how they could work with the offered technologies. [month 28]

D6.8) Sustainability plans for the HUC activities: The HUC activities must plan for a sustainable future – either
within their own community, as part of the generic production infrastructure, or through some other mechanism.
The sustainability plan for each task within the activity will be defined in this public report by assessing the
progress made to date and providing plans for the next year. [month 33]

D6.9) Annual Report on the HUC Tools and Services: A public report describing the current status of the tools
and services provided within SA3, their activity over the preceding year, and the future activities including
measures to assure sustainability of the current work. [month 35]