EGI Federated Cloud Roadmap

Overview

For users

For resource providers

Infrastructure status

Site-specific configuration

Architecture

Architecture

Technology

Roadmap

FedCloud Task Force

The EGI Federated Cloud Roadmap is available at http://go.egi.eu/fedcloud-roadmap and is defined but the TCB-Cloud board.

TCB-Cloud

This board meets regularly to define the short-, mid- and long-term plans for the technology and architectural evolution of the cloud service in EGI. The current roadmap covers the 2017 period and provides hints for the upcoming years

• Chairman: E. Fernandez/EGI Foundation
• Cloud standards, OCCI, OpenNebula: B. Parak/CESNET
• Cloud standards, CDMI, data management: B. Kryza/CYFRONET
• Cloud integration modules: A. Lopez/CSIC
• VM Image Management, Cloud marketplace: M. Chatziangelou/IASA
• INDIGO DataCloud: G. Donvito/INFN
• EGI Technology and Operations: V. Spinoso/INFN, T. Ferrari/EGI Foundation
• Cloud providers:
  • M. Antonacci/INFN, RECAS (OpenStack)
  • J. Pansanel/France Grilles, IN2P3 (OpenStack);
  • V. Tran/II SAS (OpenStack)
  • K. Koumantaros/GRNET (Synnefo)

The board develops the roadmap in consultation with:

• the [EGI Federated Cloud task force]
• the User Community Board (UCB) and the team in charge of cloud support
• the Operations Management Board (OMB)
• the Security Coordination Team and AAI experts
• the Service and Solution Board (SSB)
• commercial cloud providers

Roadmap highlights

Usability and accessibility of the EGI Cloud Service

These actions seek to improve the usability and accessibility of the EGI Cloud by removing some of the identified barriers for adoption.

Name	Description	Status
AppDB VMOps	A single GUI for the EGI Cloud that allows users to manage VMs on different providers just without the need to deal with X.509 certificates and completely integrated with the AppDB Cloud Marketplace.	In beta testing for selected VOs. Available on request for new VOs
OCCI 1.2	The latest version of the OCCI standard facilitates the development of new clients and introduces new features such as snapshotting and resizing VMs.	In production for OpenStack and Synnefo, release candidate for OpenNebula
OpenID Connect Support	The support of OIDC at the resource providers will allow use of federated identity mechanisms and integration with EGI CheckIn, thus completely removing the need for user's X.509 certificates and easing the development and deployment of web portals without token translation mechanisms. OIDC also allows using federated identity to interact with Command Line Interface tools or directly with APIs	Pilot OpenStack sites under integration. Synnefo and OpenNebula support under development
IaaS Federated Access Tools	This layer of the EGI Cloud Architecture hides the underlying heterogeneity from users and provides common ways of managing different resources. A set of tools to be used in this layer needs to be investigated and documented.	First set of tools evaluated and documented

Site integration

EGI maintains a collection of open source components that facilitate the integration of different Cloud Management Frameworks into the Federation. The following table collects the main developments in this area:

Name	Description	Status
CloudKeeper	CloudKeeper is the replacement for the vmcatcher tool for managing VM Images at sites. This component has a more flexible architecture and is ready to be enhanced as needed in the future	Released into CMD, deployed in the Infrastructure
Monitoring probes	New probes are needed for better assessment of the availability and reliability of the sites. Namely: check the correct subscription to VM image lists complete checking of OCCI types use of EGI-provided images for testing VM instantiation functionality	Probes in testing phase, to be moved to production as sites are stable
Monitoring fedcloud.egi.eu VO	Monitoring is performed using ops VO, but it may not detect issues in other VOs relevant to the users	Further developments needed in the monitoring framework. Stalled
Glue 2.1 schema	The Glue2.1 schema includes several improvements for publishing missing information about the providers and the VOs supported.	First prototype implementation ready. Transition plan needs to be defined
Information discovery transport	BDII is found unsuitable for the current cloud infrastructure and alternatives need to be assessed	No progress
Accounting for Long Running VMs	Accounting information for VMs running for more than one month is incorrectly assigned the first month the VM has been running.	First implementation ready, testing scalability for going into production

EGI Federated Cloud task force

The EGI Federated Cloud task force gathers together scientific communities, R&D projects, and technology and resource providers so they can design the tools and services that support the federation of cloud providers, can share best practices, and can offer user support and training in a collaborative fashion. This enables community cloud solutions to develop faster, with a lower cost and with a more sustainable future. The task force members:

• Capture requirements from user communities needing federated cloud services .
• Identify, integrate and enhance open source tools and services that enable cloud federations for research and education.
• Develop and maintain tools and services to fill gaps in third party solutions to reach production quality cloud federations.
• Provide consultancy and training for communities on how to build a federated cloud to meet custom community demands under certain constraints.
• Provide training and support for existing and potential users of cloud federations about topics, such as how to port or develop cloud-based applications; how to operate services in the cloud, how to join a cloud federation with a service.
• Facilitate the reuse of cloud federation tools and services across participating cloud federations to lower total cost of development and to improve cloud sustainability.
• Promote Platform as a Service (PaaS) and Software as a Service (SaaS) environments that are proven to be robust and reusable across communities to interact with federated IaaS clouds.
• Provide service management and security oversight for participating clouds and cloud federations.
• Act as a discussion forum where cloud federations can be discussed and specific questions can be analysed with top-world experts.
• Organise dissemination and marketing events, workshops and conferences relating to the topics of the collaboration.

Operations

The Task Force operates on its mailing list and on a conference call. At the moment, our effort is organised in work groups that any member of the Task Force is free to lead or to contribute to. Each member of Task Force can organise a meeting dedicated to further progress a specific workgroup activity.

The Task Force meet Tuesday at 11:00 CET every two weeks in a conference call. Minutes of the meetings are publicly available in the INDICO pages.

• Mailing list (incl. Mailing list archive)
• Meeting planner

FedCloud work groups

The FedCloud Task Force activities are split across work groups. A leader is elected for each work group and members of the Task Force are free to spend their effort in one or more groups. Each work group investigates one or more capabilities that are required by a federation of clouds. The work done is recorded in the group workbench and, eventually, translated into the Task Force blueprint.

With the development of the testbed and of the blueprint, new capabilities will be investigated and addressed. As a consequence, new work groups are added to the Task Force when required.

Running

• VM Management
• Data Management
• Information Discovery
• Accounting
• Monitoring
• federated AAI
• VM Image Management
• Brokering
• Security
• Intra Cloud networking

Closed

• Notification
• Contextualisation

Join the EGI FedCloud Task Force

If you are interested in joining the EGI FedCloud Task Force, please send a mail to fedcloud Mailing list introducing yourself.