2019-bidding/datamanagement

From EGIWiki
Revision as of 14:15, 20 November 2019 by Enolfc (talk | contribs) (Technical description)
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security



EGI Services and Service Management Support menu: Bids Old Bids Performance

Go back to the EGI Services Bidding page.

Data Management Services

Introduction

EGI Data Management Services support the management of large volumes of data spread across the infrastructure following FAIR principles, including:

  • Discovery and publication of data sets to specific communities or worldwide.
  • Transfer of large sets of data.
  • Metadata management
  • Manual and policy-based replication for resiliency and availability purposes.
  • Efficient exploitation of data by bringing it close to computing.
  • On-demand cache of data to avoid full replication.

Technical description

This call welcomes any Data Management Service compatible with the EGI Computing and Storage services, able to authenticate users with Check-in and covering totally or partially the following features:

  • Digital repository for annotating, publishing, sharing, cataloguing, discovering, and using data.
  • Asynchronous and reliable file transfer across distributed infrastructures.
  • On-demand and automatic replication and caching of data.
  • Exposing data to computing services with POSIX semantics.

The Data Management service should be offered as a centrally managed instance that will be run by the project and will provide access to ‘long-tail of science’ type use cases on ‘generic clouds’ for any user, and to thematic groups on the ‘thematic clouds’. Additionally, the platform provider should be capable of setting up dedicated instances to specific communities as needed upon request. These instances must include support for monitoring via the EGI Monitoring service and provide accounting records as specified by the EGI Accounting service.

Example of technologies that power services that may apply to this call are: Invenio, CKAN, FTS, Onedata, Rucio.

Coordination

This activity is responsible for the coordination of the service maintenance activities with EGI operations team and other technology providers for the EGI Core Infrastructure.

Operations

  • Daily running of the service instances.
  • Provisioning of a high availability configuration.
  • Creating an Availability and Continuity Plan and implementing countermeasures to mitigate the risks defined in the related risk assessment.

Maintenance

This activity includes:

  • Bug fixing, proactive maintenance, improvement of the system.
  • Maintenance of probes to test the functionality of the service.
  • Documentation.

Software Compliance

  • Unless explicitly agreed, software being used and developed to provide the service should:
    • Be licensed under an open source and permissive license (like MIT, BSD, Apache 2.0,...).
      • The license should provide unlimited access rights to the EGI Foundation and EGI federation member organisations.
    • Have source code publicly available via a public source code repository (if needed a mirror can be put in place under the EGI organisation in GitHub.) All releases should be appropriately tagged.
    • Adopt best practices:
      • Defining and enforcing code style guidelines.
      • Using Semantic Versioning.
      • Using a Configuration Management frameworks such as Ansible.
      • Taking security aspects into consideration through at every point in time.
      • Having automated testing in place.
      • Using code reviewing.
      • Treating documentation as code.
        • Documentation should be available for Developers, administrators and end users.

IT Service Management compliance

  • Key staff who deliver services should have foundation or basic level ITSM training and certification.
    • ITSM training and certification could include FitSM, ITIL, ISO 20000 etc.
  • Key staff and service owners should have advanced/professional training and certification covering the key processes for their services.
  • Providers should have clear interfaces with the EGI SMS processes and provide the required information.
  • Providers should commit to improving their management system used to support the services they provide.

Support

  • Support is provided through the EGI Helpdesk.
  • Support hours: eight hours a day, Monday to Friday – excluding public holidays of the hosting organisation(s).

Service level targets

Catch-all instance must have at least 95% A/R on a monthly basis. Dedicated instances A/R targets will be defined in additional SLA/OLA.

Response to ticktes in EGI Helpdesk within support hours: Medium.

Application template

Applications must follow the template available at XXX

Selection criteria

TO BE CLARIFIED

In addition to the above requirements with respect to features, Software and ITSM compliance, the following criteria will be taken into account as part of the selection process:

  • TRL level
  • Support level
  • Adoption of the service by existing communities
  • Level of integration with EGI services
  • Interoperability with other e-Infrastructures
  • Cost
  • Features roadmap
  • Long-term sustainability