Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Federated Cloud VM Management

From EGIWiki
Jump to navigation Jump to search
Overview For users For resource providers Infrastructure status Site-specific configuration Architecture



Scenarios: Federated AAI Accounting VM Image Management Brokering IntraCloud Networking
Monitoring VM Management Data Management Information Discovery Security


Scope

  • Review available Open Standards for Virtual Machine Management.
  • Choose an appropriate Open Standard to handle Virtual Machine Management in EGI Federated Cloud.
  • (if necessary) Implement support for the chosen Open Standard in/for as many Cloud Management Frameworks (CMFs)/Public Cloud Service Providers (PCSP) as possible.
  • (if necessary) Implement/integrate client(s) harnessing all relevant available features of the chosen Open Standard.
  • Deploy and continuously maintain deployed components.
  • Provide EGI Operations with extensive installation documentation.
  • Provide EGI UCST with extensive user documentation.
  • Provide feedback to organization(s) developing the chosen Open Standard and help with its future development.

Members

Role Institution Name
Scenario leader CESNET Boris Parák
Collaborator CESNET Zdeněk Šustr
Collaborator IFCA Enol Fernández
Collaborator GRNET Kostas Koumantaros

Roadmap

Interoperability

Aligning behavior between different implementations and CMFs.

See Federated_Cloud_VM_Management#Interoperability_Table

Authentication

Introducing the support for Keystone-like authentication protocol in every available implementation and CMF.

See Keystone-VOMS docs

Deployment

Continuous deployment of maintenance updates and new versions for every implementation and CMF.

Support for new platforms & public providers

Introducing support for new CMFs and public cloud providers.

Support for native interfaces

Exploring the possibility of exploiting native CMF interfaces as well as inter-operable interfaces.


Documentation

Available Open Standards

OCCI

Developed by the Open Grid Forum, OCCI is a boundary protocol and API that acts as a service front-end to a provider’s internal management framework by exposing its resources. As of this writing (early 2015) the specification consist of three documents:

  • OCCI Core describing the formal definition of the the OCCI Core Model
  • OCCI Infrastructure defining the OCCI Infrastructure extension for the IaaS domain, defining additional resource types, their attributes and the actions that can be taken on each resource type
  • OCCI HTTP Rendering defining interaction with the OCCI Core Model using the RESTful OCCI API

The OCCI Working Group is also working on additional specification, currently in various stages of progress:

  • OCCI XML Rendering
  • OCCI JSON Rendering
  • OCCI Billing and Monitoring
  • OCCI PaaS extension
  • OCCI SLAs extension

CIMI

The Cloud Infrastructure Management Interface is a standard released by DMTF. Similarly to OCCI, it also consists of multiple specifications:

  • CIMI Primer (DSP2027) v. 1.0.0
  • CIMI (DSP0263) v. 1.1.0
  • CIMI-CIM (DSP0264) v. 1.0.0

CIMI is a highly specific IaaS protocol, defining a wide range of attributes applicable in the context of IaaS. This is what mainly distinguishes it from OCCI, which is a much more light-weight, generic boundary-level protocol, relying on its extensibility to cover specific areas.

OGF's Open Cloud Computing Interface

OCCI was selected for the FedCloud environment for multiple reasons, chief among them its simplicity, extensibility and the fact that other standards of that kind (such asi CIMI) have not been published until well after the initiative started.

For protocol details, please, read:

  • OGF OCCI 1.1 Core Specification -- GFD-183
  • OGF OCCI 1.1 Infrastructure Specification -- GFD-184
  • OGF OCCI 1.1 HTTP Rendering Specification -- GFD-185

Available OCCI implementations

Server-side Implementations

rOCCI-server

rOCCI-server, part of the rOCCI Framework, is a stand-alone service providing "translation" between OCCI and proprietary cloud management frameworks, which are not otherwise OCCI-compatible. Individual types of clod management framework are provided through specific backends. The list of currently available backends is available in the ROCCI-server_Backend_development_status page.

rOCCI-server can be obtained as source from GitHub or in the form of packages from EGI's AppDB.

Common documentation for the whole rOCCI framework is available at the EGI Wiki.

OCCI-OS

OCCI-OS provides a python egg which can be easily deployed in OpenStack and will thereby add OCCI support and interface to OpenStack. Available and documented at [1].

snf-occi

snf-occi implements the OCCI specification and maps it to the Synnefo OpenStack API, so it acts as a translation bridge between OCCI and Synnefo, providing an OCCI compatibility layer. [2]

Client-side Implementations

rOCCI-cli

rOCCI-cli, another part of the rOCCI Framework, is a command-line client capable of communicating with any OCCI-enabled server. It supports several methods of authentication, including X.509 certificates and proxy certificates.

rOCCI-cli can be obtained as source from GitHub or in the form of packages from EGI's AppDB.

Common documentation for the whole rOCCI framework is available at the EGI Wiki.

Libraries

rOCCI-core and rOCCI-api

Core parts of the rOCCI framework, the rOCCI-core library implements the OCCI class structure in Ruby, which allows developers to work with OCCI concepts natively. rOCCI-api takes care of transporting OCCI messages, currently supporting HTTP as the only transport protocol.

pyssf

pyssf is a set of python modules developed by Platform Computing, man contributors to the OCCI standard. Source as well as documentation is available from GitHub [3].

jOCCI-core and jOCCI-api

jOCCI is an independent OCCI implementation by the authors of rOCCI. It satisfies the demand for a Java library, replacing the previous alternative based on JRuby. Currently in development, is is available as source from Github [4] [5].

Interoperability Table

Legend
Symbol Meaning
OK Compliant with the official specification and EGI Federated Cloud requirements/extensions.
PART Partially compliant with the official specification and EGI Federated Cloud requirements/extensions. Details available in Comments.
NOK Not compliant with the official specification and EGI Federated Cloud requirements/extensions. Details available in Comments.
UNKN Compliance with the official specification and EGI Federated Cloud requirements/extensions is unknown.

Servers

Please note that this table is work in progress!

Compatibility of different OCCI server implementations – this table sums up basic tasks that can be performed over OCCI in different cloud management frameworks
Task Subtask rOCCI-server OCCI-OS
OpenStack
snf-occi
Synnefo
Comments
OpenNebula AWS MS Azure
Retrieving model Category Classes
(Kind, Action, Mixin)
OK OK NOK OK OK
Core Kinds
(Entity, Resource, Link)
OK OK NOK OK OK
Infrastructure Kinds
(Compute, Network, Storage)
OK OK NOK OK PART * Synnefo -- Storage is not exposed
Infrastructure Mixins
(os_tpl, resource_tpl)
OK OK NOK OK OK
Filtering On Query Interface UNKN UNKN UNKN UNKN UNKN
On Collection UNKN UNKN UNKN UNKN UNKN
Creating Compute OK OK NOK OK OK
Querying Compute OK OK NOK OK OK
Destroying Compute OK OK NOK OK OK
Creating Storage OK OK NOK OK UNKN
Querying Storage OK OK NOK OK UNKN
Destroying Storage OK OK NOK OK UNKN
Attaching Storage to Compute On creation OK OK NOK OK UNKN
In run-time OK OK NOK OK UNKN
Detaching Storage from Compute OK OK NOK OK UNKN
Creating Network PART NOK NOK NOK UNKN * rOCCI-server/OpenNebula -- requires platform-specific and infrastructure-specific knowledge
Querying Network OK NOK NOK PART UNKN * OCCI-OS includes two hardcoded networks, no real query to the underlying OpenStack
Destroying Network OK NOK NOK NOK UNKN
Attaching Network to Compute On creation OK OK NOK NOK UNKN
In run-time OK OK NOK OK UNKN
Detaching Network from Compute OK OK NOK OK UNKN
Actions on Resources Compute – start OK OK NOK OK UNKN
Compute – stop OK OK NOK OK UNKN
Compute – restart OK OK NOK OK UNKN
Compute – suspend OK NOK NOK OK UNKN
Storage – online OK NOK NOK NOK UNKN
Storage – offline OK NOK NOK NOK UNKN
Storage – backup OK NOK NOK NOK UNKN
Storage – snapshot NOK OK NOK OK UNKN
Storage – resize NOK NOK NOK NOK UNKN
Network – up NOK NOK NOK NOK UNKN
Network – down NOK NOK NOK NOK UNKN
Contextualization on Compute OK OK NOK OK UNKN

Clients

Please note that this table is work in progress!

Compatibility of different OCCI client implementations – this table sums up basic tasks that can be performed over OCCI in different clients
Task Subtask rOCCI-{core, api, cli} jOCCI-{core, api} Comments
Retrieving model Category Classes
(Kind, Action, Mixin)
OK OK
Core Kinds
(Entity, Resource, Link)
OK OK
Infrastructure Kinds
(Compute, Network, Storage)
OK OK
Infrastructure Mixins
(os_tpl, resource_tpl)
OK OK
Filtering On Query Interface NOK NOK
On Collection NOK NOK
Creating Compute OK OK
Querying Compute OK OK
Destroying Compute OK OK
Creating Storage OK OK
Querying Storage OK OK
Destroying Storage OK OK
Attaching Storage to Compute On creation OK OK
In run-time OK OK
Detaching Storage from Compute OK OK
Creating Network OK OK
Querying Network OK OK
Destroying Network OK OK
Attaching Network to Compute On creation OK OK
In run-time OK OK
Detaching Network from Compute OK OK
Actions on Resources OK OK
Contextualization on Compute OK OK

Installation Manuals

OpenNebula

See MAN10#OpenNebula

OpenStack

See MAN10#OpenStack

Synnefo

See Fedcloud-tf:WorkGroups:Scenario1:SynnefoInstallation

Amazon WS EC2

See Fedcloud-tf:WorkGroups:Scenario1:AWSEC2Installation

Microsoft Azure

See Fedcloud-tf:WorkGroups:Scenario1:MSAzureInstallation

References