Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

User:Enolfc

From EGIWiki
Jump to navigation Jump to search

Enol Fernandez - CSIC https://www.egi.eu/sso/userDetail/enolfc


Overview For users For resource providers Infrastructure status Site-specific configuration Architecture




EGI Cloud Federation

The EGI Federated Cloud is a multi-national cloud system that integrates institutional clouds into a scalable computing platform for data and/or compute driven applications and services. The initial architecture of the EGI Federated Cloud was defined in 2011-2012 and was fully implemented by May 2014. Currently, the federation is a collaboration that enables various types of cloud federations to serve diverse demands of researchers from both academia and industry. The EGI Federated Cloud brings together scientific communities, R&D projects, technology and resource providers to form a community that integrates and maintains a flexible solution portfolio that enables various types of cloud federations with IaaS, PaaS and SaaS capabilities. The collaboration is committed to the use of open source tools and services that are reusable across scientific disciplines. These tools and services form a flexible portfolio from which a scientific community can mix and match items to establish its own, customised cloud federation.


The EGI Federated Cloud provides the services and technologies to create federation of clouds (community, private or public clouds) that operate according to the preferences, choices and constraints set by its members and users. The EGI Cloud Federations are modelled around the concept of an abstract Cloud Management stack subsystem that is integrated with components of the EGI Core Infrastructure and that provides a set of agreed uniform interfaces within the community it provides services to.

Federated Cloud Model

The EGI Cloud Federation (see Figure) is a hybrid cloud composed by public, community and private clouds, all supported by the EGI Core Infrastructure Platform services. The EGI Federated Cloud is composed by multiple “realms”, each realm having homogeneous cloud management interfaces and capabilities. A cloud realm is a subset of cloud providers exposing homogeneous cloud management interfaces and capabilities. The Open Standards Cloud Realm supports the usage of open standards for its interfaces and is completely integrated with the EGI Core Infrastructure Platform. A Community Platform provides community-specific data, tools and applications, which can be supported by one or more realms.

Services in cloud federations

Despite the large diversity in the type of cloud realms, a relatively small number of identical building blocks (or federator services) can be identified in almost all of them. These services turn individual clouds into a federation. The table collects these common services to help architects identify topics they should focus on when designing a cloud federation. Technical details fro these are also available at Federated Cloud Technology.

Federation Service Role within the federation Existing technical solution in EGI
Service Registry A registry where all the federated sites and services are registered and state their capabilities. The registry provides the ‘big picture view’ about the federation for both human users and online services (such as service monitors). GOCDB
Information System A database that provides real-time view about the actual capabilities and load of federation participants. Can be used by both human users and online services. BDII
Virtual Machine Image Catalogue A catalogue of Virtual Machine Images (VMIs) that encapsulate those software configurations that is useful and relevant for the given community (typically pre-configured scientific models and algorithms). AppDB
Image replication mechanism A system that automatically replicates VMIs from the federation VMI catalogue to each of the member sites, as well as removes them when needed. Automated replication can ensure consistency of capabilities across sites and is very often coupled with a VMI vetting process to ensure that only properly working, and relevant VMIs are replicated to the cloud sites of the community. vmcatcher/vmcaster
Single sign-on for users Ensuring that users of the federation need to register for access only once before they can use the federated services. Single sign-on is increasingly implemented in the form of identity federations in both industry and academia. IGTF X509 proxies with VOMS extensions
Integrated view about resource/service usage A system that pulls together usage (accounting) information from the federated sites and services, integrates the data and presents them in such a way that both individual users and communities can monitor their own resource/service usage across the whole federation. Cloud Usage Record, APEL Accounting repository and portal
Integrated interfaces or user environments Having interfaces through which users and user applications can interact with the services offered by the various cloud providers. In case of an IaaS cloud federation these interfaces offer compute, storage and network management capabilities. OCCI API and OpenStack API
Availability Monitoring Use a shared system to monitor and collect availability and reliability statistics about the distributed cloud service providers and to retrieve this information programmatically. ARGO monitoring system
Federated service management tools A set of processes, policies, activities and supporting tools customized to the federated cloud. EGI federated service management

EGI cloud realms

The EGI Federated Cloud can support multiple cloud federations (community specific, private or public). Based on the EGI federation services and custom external solutions, any scientific community can create a federated cloud. Each community or e-infrastructure that wants to build a cloud federation decides the services required to support their computational needs. Because these cloud federations are largely built from tools and services of the same solution portfolio, they can maintain the portfolio together; they can share best practices, and can offer user support and training in a collaborative fashion.

EGI currently operates two realms: the Open Standards Realm and the OpenStack Realm. Both are completely integrated with the EGI federator services described above but use different interfaces to offer the IaaS capabilities to the users: the Open Standards Realm uses OCCI standard (supported by providers with OpenNebula, OpenStack or Synnefo cloud management frameworks), while the OpenStack Realm uses OpenStack native Nova API (support limited to OpenStack providers). This OpenStack Realm was introduced in the federation during November 2015 and most of the resource providers already in the Open Standards Realm using OpenStack have started to provide this API along with the existing OCCI interface.

Service Open Standards Realm OpenStack Realm
Service Registry GOCDB
Single sign-on X.509 proxies with VOMS extensions
Accounting Cloud Usage Record
Information discovery BDII
VM Image catalogue AppDB
VM Image distribution vmcatcher/vmcaster
IaaS interface OCCI OpenStack Compute API
Monitoring ARGO (OCCI specific probes) ARGO (OpenStack specific probes)