2019-bidding/umd-cmd-infra

Main

EGI.eu operations services

Support

Documentation

Tools

Activities

Performance

Technology

Catch-all Services

Resource Allocation

Security

EGI Services and Service Management Support menu:

Bids •

Old Bids •

Performance •

Go back to the EGI Services Bidding page.

Service name: UMD Software provisioning infrastructure

(to add requirements on software licenses and fitsm training&certification)

Introduction

The software provisioning infrastructure provides the technical tools to support the UMD release process, used for UMD (Unified Middleware Distribution) and CMD (Cloud Middleware Distribution) from pulling packages from the developers repositories to the build of a release. CMD follow exactly the same process as for UMD; the only difference is in the content (CMD contains cloud-oriented software) and in the release cycle (months for CMD against years for UMD).

Technical description

The software provisioning infrastructure is composed by the following components:

• Integration with RT, a new product release (the tuple Product, Platform, Architecture) is associated with a RT ticket, which tracks the status of the product in the software provisioning process
• Submission of new products with XML.
• Repository Back-end: responsible unit for handling the movement of packages between repositories, validating the individual product releases submissions, building accumulative as well as per-product YUM/APT repositories (multiple per OS/Arch case) and the other automations needed to perform the UMD and CMD operations. It also provide a RESTful API for external integrations (e.g. with the UMD and CMD portal/frontend)
• Composer: a web-based interface for bundling versioned software products that have successfully passed the UMD and CMD verification process, into a robust UMD and CMD release ready to be deployed either to the production or the candidate repositories.
• ReleaseXML editor: a web interface to create a new entry in the UMD and CMD release process, it is connected with the RT and the bouncer
• Repositories: the following repositories must be maintained for every operating system and major release supported:
  • Untested: contains the packages to be installed during the verification
  • Testing: contains the packages to be installed during staged rollout
  • Base: contains the packages released in the first major release
  • Update: contains the packages released in the update releases
  • Release Candidate: it is generated before a UMD and CMD release, to simulate the production repositories after the UMD and CMD release under preparation. This is used to test the installability of the newly released components, as well as the products already in production.
• The processes to move products between repositories and to create releases must be as automated as possible.
• The task must provide statistics about the repository usage in terms of downloads, aggregated by packages and time.
• Front-end, the information about UMD and CMD releases (release notes, list of components, configuration) must be available in a web frontend.

Note: the architecture of the internal components is not mandatory, but the services provided must be equivalent.

The software provisioning infrastructure must support multiple distributions, multiple operating system (EL based, and Debian based) and major releases (at least two major releases).

The infrastructure should also support a “Preview” repository where products are quickly released without verification. It is not not an official UMD repository, but it represents a place where products can be made available to service providers more quickly and directly, bypassing the quality assurance steps.

Coordination

The task must coordinate with the UMD and CMD quality assurance task as well as EGI Operations when necessary, and with the AppDB provider to support the community repositories.

Operations

The task must operate all the technical services described before:

• Repositories (production, testing, untested and RC, community repositories)
• Repositories back-end (including UMD composer)
• Web pages (repository front-end, Release XML editor)

The task must support the creation of the distributions and for each distribution the creation of the releases, creating the release candidates and the actual releases.

Maintenance

• Requirements gathering
• Documentation

Software Compliance

• Unless explicitly agreed, software being used and developed to provide the service should:
  • Be licensed under an open source and permissive license (like MIT, BSD, Apache 2.0,...).
    • The license should provide unlimited access rights to the EGI Foundation and EGI federation member organisations.
  • Have source code publicly available via a public source code repository (if needed a mirror can be put in place under the EGI organisation in GitHub.) All releases should be appropriately tagged.
  • Adopt best practices:
    • Defining and enforcing code style guidelines.
    • Using Semantic Versioning.
    • Using a Configuration Management frameworks such as Ansible.
    • Taking security aspects into consideration through at every point in time.
    • Having automated testing in place.
    • Using code reviewing.
    • Treating documentation as code.
      • Documentation should be available for Developers, administrators and end users.

IT Service Management compliance

• Key staff who deliver services should have foundation or basic level ITSM training and certification.
  • ITSM training and certification could include FitSM, ITIL, ISO 20000 etc.
• Key staff and service owners should have advanced/professional training and certification covering the key processes for their services.
• Providers should have clear interfaces with the EGI SMS processes and provide the required information.
• Providers should commit to improving their management system used to support the services they provide.

Support

Support through the EGI Helpdesk.

Support hours: eight hours a day, Monday to Friday – excluding public holidays of the hosting organisation(s).

Service level targets

UMD repositories and web front-end, as well as the community repository, must have 90% availability and reliability on a monthly base. The other components used by the UMD team only must have 75% availability and 90% reliability during working hours.

Load Balancing and high availability configuration (between two or more servers) is required in order to handle the load on the repositories and the front-end service.

Response to incident records in EGI Helpdesk within support hours: Medium.

Effort (EGI-related activities)

Bids planning a total effort up to 5 Person Months/year should allow these services and activities to be addressed appropriately.

Effort (EOSC-related activities)

Partners are encouraged to submit details of activities and proposed costing of effort for EOSC related activities. This may include activities related to development of new functionality required by EOSC communities in addition to activities delivering services to these communities.