|
|
| Line 1: |
Line 1: |
| {{Fedcloud_Menu}} {{FedCloud_Innovation_Menu}} {{TOC_right}}
| | #REDIRECT [[Federated Cloud Architecture]] |
| | |
| The EGI Federated Cloud Roadmap is available at http://go.egi.eu/fedcloud-roadmap and is defined but the TCB-Cloud board.
| |
| | |
| = TCB-Cloud =
| |
| | |
| This board meets regularly to define the short-, mid- and long-term plans for the technology and architectural evolution of the cloud service in EGI. [http://go.egi.eu/fedcloud-roadmap The current roadmap] covers the 2017 period and provides hints for the upcoming years
| |
| | |
| *Chairman: E. Fernandez/EGI Foundation
| |
| *Cloud standards, OCCI, OpenNebula: B. Parak/CESNET
| |
| *Cloud standards, CDMI, data management: B. Kryza/CYFRONET
| |
| *Cloud integration modules: A. Lopez/CSIC
| |
| *VM Image Management, Cloud marketplace: M. Chatziangelou/IASA
| |
| *INDIGO DataCloud: G. Donvito/INFN
| |
| *EGI Technology and Operations: V. Spinoso/INFN, T. Ferrari/EGI Foundation
| |
| *Cloud providers:
| |
| **M. Antonacci/INFN, RECAS (OpenStack)
| |
| **J. Pansanel/France Grilles, IN2P3 (OpenStack);
| |
| **V. Tran/II SAS (OpenStack)
| |
| **K. Koumantaros/GRNET (Synnefo)
| |
| | |
| The board develops the roadmap in consultation with:
| |
| | |
| *the [EGI Federated Cloud task force]
| |
| *the User Community Board (UCB) and the team in charge of cloud support
| |
| *the Operations Management Board (OMB)
| |
| *the Security Coordination Team and AAI experts
| |
| *the Service and Solution Board (SSB)
| |
| *commercial cloud providers
| |
| | |
| = Short term roadmap (2017) =
| |
| | |
| == Usability and accessibility of the EGI Cloud Service ==
| |
| | |
| These actions seek to improve the usability and accessibility of the EGI Cloud by removing some of the identified barriers for adoption.
| |
| | |
| {| style="border:1px solid black; text-align:left;" class="wikitable sortable" cellspacing="0" cellpadding="5"
| |
| |- style="background:lightgray;"
| |
| ! style="border-bottom:1px solid black;" | Name
| |
| ! style="border-bottom:1px solid black;" | Description
| |
| ! style="border-bottom:1px solid black;" | Status
| |
| |-
| |
| | '''AppDB VMOps'''
| |
| | A single GUI for the EGI Cloud that allows users to manage VMs on different providers just without the need to deal with X.509 certificates and completely integrated with the AppDB Cloud Marketplace.
| |
| | [https://dashboard.appdb.egi.eu/vmops In production for selected VOs]. Available on request for new VOs
| |
| |-
| |
| | '''OCCI 1.2'''
| |
| | The latest version of the OCCI standard facilitates the development of new clients and introduces new features such as snapshotting and resizing VMs.
| |
| | In production for OpenStack and Synnefo, release candidate for OpenNebula
| |
| |-
| |
| | '''OpenID Connect Support'''
| |
| | The support of OIDC at the resource providers will allow use of federated identity mechanisms and integration with EGI CheckIn, thus completely removing the need for user's X.509 certificates and easing the development and deployment of web portals without token translation mechanisms. OIDC also allows using federated identity to interact with Command Line Interface tools or directly with APIs
| |
| | Pilot OpenStack and Synnefo sites under integration. Release candidate for OpenNebula available.
| |
| |-
| |
| | '''IaaS Federated Access Tools'''
| |
| | This layer of the [[Federated_Cloud_Architecture#Federated_IaaS|EGI Cloud Architecture]] hides the underlying heterogeneity from users and provides common ways of managing different resources. A set of tools to be used in this layer needs to be investigated and documented.
| |
| | [[Federated Cloud IaaS Orchestration|First set of tools evaluated and documented]]
| |
| |-
| |
| | '''Application portability'''
| |
| | Users have difficulties on porting applications from one provider to another due to the big heterogeneity of the site configuration (especially in network).
| |
| | First survey of site configurations available. Need to improve discoverability and automatic handling of the heterogeneity.
| |
| |}
| |
| | |
| == Site integration ==
| |
| | |
| EGI maintains a collection of open source components that facilitate the integration of different Cloud Management Frameworks into the Federation. The following table collects the main developments in this area:
| |
| | |
| {| style="border:1px solid black; text-align:left;" class="wikitable sortable" cellspacing="0" cellpadding="5"
| |
| |- style="background:lightgray;"
| |
| ! style="border-bottom:1px solid black;" | Name
| |
| ! style="border-bottom:1px solid black;" | Description
| |
| ! style="border-bottom:1px solid black;" | Status
| |
| |-
| |
| | '''CloudKeeper'''
| |
| | CloudKeeper is the replacement for the vmcatcher tool for managing VM Images at sites. This component has a more flexible architecture and is ready to be enhanced as needed in the future
| |
| | Released into CMD, deployed in the Infrastructure
| |
| |-
| |
| | '''Monitoring probes'''
| |
| | New probes are needed for better assessment of the availability and reliability of the sites. Namely:
| |
| * check the correct subscription to VM image lists
| |
| * complete checking of OCCI types
| |
| * use of EGI-provided images for testing VM instantiation functionality
| |
| * OIDC support for probes
| |
| | Probes in testing phase, to be moved to production as sites are stable.
| |
| OIDC support under development
| |
| |-
| |
| | '''Monitoring fedcloud.egi.eu VO'''
| |
| | Monitoring is performed using ops VO, but it may not detect issues in other VOs relevant to the users
| |
| | Further developments needed in the monitoring framework. Stalled
| |
| |-
| |
| | '''Glue 2.1 schema'''
| |
| | The Glue2.1 schema includes several improvements for publishing missing information about the providers and the VOs supported.
| |
| | First prototype implementation ready. Transition plan needs to be defined
| |
| |-
| |
| | '''Information discovery transport'''
| |
| | BDII is found unsuitable for the current cloud infrastructure and alternatives need to be assessed
| |
| | New pilot information infrastructure from IASA under development.
| |
| |-
| |
| | '''Accounting for Long Running VMs'''
| |
| | Accounting information for VMs running for more than one month is incorrectly assigned the first month the VM has been running.
| |
| | First implementation ready, testing scalability for going into production
| |
| |}
| |
| | |
| = Long term roadmap (2018 onwards) =
| |
| | |
| == New services ==
| |
| | |
| TCB-Cloud has identified a set of new services that could attract new communities and/or facilitate the porting of applications for existing ones.
| |
| | |
| {| style="border:1px solid black; text-align:left;" class="wikitable sortable" cellspacing="0" cellpadding="5"
| |
| |- style="background:lightgray;"
| |
| ! style="border-bottom:1px solid black;" | Service
| |
| ! style="border-bottom:1px solid black;" | Description
| |
| ! style="border-bottom:1px solid black;" | Requirements
| |
| |-
| |
| | '''Filesystem as a Service (FSaaS)'''
| |
| | Create shareable filesystems within one site that can scale as needed
| |
| | Users with needs to share data between VMs but no real need for federation of the data (e.g. running MPI applications)
| |
| Access to legacy filesystems at sites (e.g. GPFS, Lustre)
| |
| |-
| |
| | '''Container orchestration as a Service'''
| |
| | Managed Kubernetes/Swarm/Mesos with elasticity
| |
| |
| |
| Communities with docker-based applications (e.g. ICOS)
| |
| |-
| |
| | '''Data Analytics as a Service'''
| |
| | Managed Hadoop/Spark/Storm/Galaxy/Jupyter.... with elasticity
| |
| | Several communities already requesting these services and some of them already integrated in AoD
| |
| |-
| |
| |'''HTC cluster as a Service'''
| |
| | Managed HTC Cluster (SGE, Slurm, Mesos, Torque) with elasticity
| |
| | Facilitate migration of existing use cases to the cloud
| |
| |-
| |
| | '''VPN as a Service'''
| |
| | Enable creation of VPNs easily among members of the federation
| |
| | Improved security of deployments by reducing risk surface
| |
| Remove the need for large pools of public IPs at operators
| |
| Sensitive data handling use cases
| |
| |-
| |
| | '''DNS as Service'''
| |
| | Enable DNS management by users, provide names to VMs independent of the IP
| |
| | Several providers facing issues with existing users requesting this kind of feature
| |
| |-
| |
| | '''LB as a Service'''
| |
| | Load Balancing for large-scale applications
| |
| | Simplify the use of distributed infrastructure
| |
| |-
| |
| | '''HPC as a Service'''
| |
| | Bare-metal like performance for IaaS clouds
| |
| Pluggable GPUs, FPGAs
| |
| | Enable HPC applications in the cloud (meteo, Terradue)
| |
| |-
| |
| |}
| |
| | |
| == Site integration ==
| |
| | |
| The core components that enable the EGI Federated Cloud need to be maintained and evolved to maintain compatibility with the resource providers technology and add new features as requested by the user communities. As general activity, the components are expected to:
| |
| * Improve their programmability, providing complete APIs specification in adequate format for facilitating the generation clients (e.g. following the OpenAPI initiative) and Swagger.
| |
| * Lower the barriers to integrate and operate resource centres in the federation by a) minimizing the number of invasive components used; b) contributing code to upstream distributions; and c) use only public APIs of the Cloud Management Frameworks.
| |
| | |
| The table below lists the main identified developments needed on each component:
| |
| | |
| {| style="border:1px solid black; text-align:left;" class="wikitable sortable" cellspacing="0" cellpadding="5"
| |
| |- style="background:lightgray;"
| |
| ! style="border-bottom:1px solid black;" | Component
| |
| ! style="border-bottom:1px solid black;" | Evolution
| |
| |-
| |
| | '''Cloud information provider'''
| |
| |
| |
| * New transport mechanisms to better handle dynamic information.
| |
| * Expand sources of information (data, quota information, …)
| |
| * Handle information that is user/VO specific (e.g. quota)
| |
| |-
| |
| | '''CloudKeeper'''
| |
| | * Provide mechanisms to distribute images without delays
| |
| |-
| |
| | '''AppDB Cloud Marketplace'''
| |
| |
| |
| * Integration with security tools, better control of images
| |
| * Distribution of software in docker containers (INDIGO)
| |
| * Provide mechanisms to distribute images almost immediately
| |
| * Endorser dashboard
| |
| * Security dashboard
| |
| * VO-wide image list dashboard
| |
| * Docker repository
| |
| |-
| |
| | '''AppDB VMOps'''
| |
| |
| |
| * Improve contextualization support
| |
| * Definition of complex topologies (e.g. using IaaS Orchestrator templates)
| |
| |-
| |
| | '''AAI'''
| |
| |
| |
| * Remove X.509 for users and sites
| |
| * Complete support for OIDC at the providers and clients
| |
| * Delegation of credentials
| |
| * User provisioning
| |
| * User suspension/banning
| |
| * Better AuthZ control
| |
| |-
| |
| | '''Monitoring probes'''
| |
| | *New probes for any new services developed
| |
| |-
| |
| | '''Accounting'''
| |
| |
| |
| * Public IP address accounting
| |
| * Storage accounting
| |
| * Handling of pricing information
| |
| |-
| |
| | '''IaaS Federated Access Tools'''
| |
| |
| |
| * Evaluation of TOSCA as possible common language for tools
| |
| * Better AAI integration
| |
| * Elasticity Control
| |
| * Provision of templates for commonly used tools
| |
| * Brokering tools for moving computation near data
| |
| |-
| |
| | '''GPGPU support'''
| |
| |
| |
| * Complete integration of GPGPUs on FedCloud (accounting, information discovery)
| |
| |-
| |
| |}
| |
| | |
| | |
| | |
| | |
| [[Category:Federated_Cloud]]
| |
The wiki is deprecated and due to be decommissioned by the end of September 2022.