Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Fedcloud-tf:WorkGroups:Brokering"

From EGIWiki
Jump to navigation Jump to search
(Redirected page to Federated Cloud Brokering)
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}
#REDIRECT[[Federated_Cloud_Brokering]]
 
<font color="red">Leader: Alvaro Simon, JR1 </font>
 
== Collaborators  ==
 
{| border="1"
|-
! Role
! Institution
! Name
|-
| Collaborator
| OeRC
| Matteo Turilli
|-
| Collaborator
| BSC
| Daniele Lezzi
|}
 
== Scope  ==
 
This workgroup deals with the issues around cloud brokering. With 10+ resource providers lined up to be federated into the EGI cloud testbed, users need effective ways to access cloud resources. The goal is for a user to have a choice between a unified, abstracted view of the cloud testbed as a whole and the opportunity to target specific providers for their needs. As a consequence, this workgroup is concerned with both brokers and OCCI clients.
 
== Roadmap  ==
 
*Collect information about existing solutions for cloud brokering compatible with the OCCI and CDMI management interfaces;
*Collect information about existing solutions for OCCI and CDMI clients;
*Make an inventory of the available solutions;
*Choose a broker and a client for the testbed;
*If possible, join the development teams of the chosen solutions to the task force;
*Coordinate with the TF Resorce Providers to deploy the chosen solutions;
*Extend the demo testbed addressing a real-life use case.
 
== Clients Comparison Table  ==
 
{| cellspacing="0" cellpadding="5" width="90%" class="wikitable" style="border:1px solid black;"
|- style="background-color:darkgray;"
! Client/API
! URL
! OS
! Support
! OCCI/CDMI
! Functionalities
! Effort required
! Comments
|-
| Hybridfox
| http://code.google.com/p/hybridfox/
| Win/OS X/Linux
| AWS/Eucalyptus/OpenStack/OpenNebula/HP Cloud
| <span style="color:red"> '''NO'''</span>
| Manage Images/Instances/Elastic IPs/Security Groups/Key-pairs
| <span style="color:green"> '''Low'''</span>
| Hybridfox does not support x509 auth, it uses EC2 auth for OpenNebula and OpenStack
|-
| DeltaCloud
| http://deltacloud.apache.org
| Win/OS X/Linux
| EC2/Eucalyptus/OpenStack/OpenNebula/vSphere
| <span style="color:red"> '''NO'''</span>
| Create/Start/Stop/Reboot/Destroy instances
| <span style="color:orange"> '''Medium'''</span>
| DeltaCloud uses its own delta-cloud driver for each framework instead of OCCI. It provides storage support for S3, Warlus, Azure and Google Storage.
|-
| Aeolus
| http://aeolusproject.org/about.html
| Win/OS X/Linux
| Same as DeltaCloud
| <span style="color:red"> '''NO'''</span>
| Create/Start/Stop/.. instances. Manage different instances and Images from different private, public, or hybrid cloud providers.
| <span style="color:orange"> '''Medium'''</span>
| Aeolus uses DeltaCloud cross-cloud abstraction library and it includes some extra functionalities. It includes Aeolus Conductor/Composer/Orchestrator/HA Manager
|-
| rOCCI-cli
| https://github.com/gwdg/rOCCI-cli
| Linux
| OpenNebula (rOCCI-server)/OpenStack (OCCI-OS)/Synnefo (snf-occi)
| <span style="color:green"> '''YES'''</span>
| Create/Start/Stop/Reboot/Destroy instances. Dynamic linking, actions on resources, contextualization. Supports X.509 and VOMS auth.
| <span style="color:green"> '''Low'''</span>
| rOCCI-cli is an OCCI 1.1 client implementation compatible with multiple server implementations.
|}
 
== Cloud Brokering Solutions  ==
 
=== Resource Brokers  ===
 
{| cellspacing="0" cellpadding="5" width="90%" class="wikitable" style="border:1px solid black;"
|- style="background-color:darkgray;"
! Name
! URL
! Cloud SW Support
! OCCI
! External API
! Functionalities
! Effort required
! Comments
|-
| CompatibleOne
| http://www.compatibleone.org/
| OpenStack, OpenNebula, Azure, Vcloud
| Yes, supports OCCI, but implements his own OCCI interface for each of the stacks (PROCCI).
|
| Accounting, Brokering, User management, Monitoring.
| <span style="color:red"> '''High'''</span>
|
*Compatible One is a complete plattform, with its own user management, accounting and monitoring, so it overlaps in some aspects with the ongoing work.
*It does not expose an OCCI api, the user has to write its own XML files and send them to the broker.
*The credentials for each of the connectors and providers have to be configured in advance by the administrator of the broker service.
*The user has to specify in the manifests the cloud account that he wants to use, and the specific name of the image in the site. It seems that the purpose of CompatibleONE is to deploy and take care of a complete manifest description: A user wants 3 machines of type A in site FOO, and 2 machine of type B in site BAR, so he explicitly defines it in his manifests and CompatibleOne will deploy it on them.
 
|-
| VMDIRAC
| https://github.com/DIRACGrid/VMDIRAC/wiki
| &nbsp; OpenNebula, OpenStack, CloudStack and Amazon EC2
| Yes, it supports rOCCI, OpenNebula "OCCI 0.8", Amazon EC2, OpenStack Nova 1.1 and CloudStack 2 APIs.
|
| Accounting, monitoring, brokering, scheduling, HEPiX and SSH based VM contextualisation, [https://github.com/DIRACGrid/VMDIRAC/wiki#vm-horizontal-auto-scaling-setup VM Horizontal Auto-scaling].
| <span style="color:orange"> '''Medium'''</span>
|
*rOCCI support with X509/VOMS auth.
*In production: LHCb (OpenStack and OpenNebula) Belle (Amazon), Work in progress France Federated Cloud.
*Web UI including VM browsing with VM monitoring, and VM Overview Plots (Load, RunningVM, RunnigVM by Endpoint, started jobs, data transfer, transfer files)
*The number of VMs is dynamically adjusted to the current workload.
*Automatic selection of the VMs types depending on the tasks constraints.
*No necessary to install any pluging in the Cloud Managers, works with standard interfaces.
*Separated user credentials are used for job running and VM operator credentials for VM management.
*VMDIRAC developers are willing to collaborate actively with Fedcloud Taskforce.
 
|-
| SlipStream™
| http://sixsq.com/products/slipstream.html
| StratusLab, OpenStack, Abiquo, CloudSigma and Amazon EC2
| Supports OCCI via rOCCI-cli integration
| EC2 via bridge (experimental), proprietary SlipStream REST APIs
| VM image creation and deployment, contextualisation, VM cluster automated deployments.
| <span style="color:orange"> '''Medium'''</span>
|
*New release 2.0 scheduled for January 2014, with Autoscaling capabilities.
*Acts as single interface and point of entrance to multiple clouds. Does not provide a brokering mechanism at application level.
*It is able to use deploy VMs across different cloud frameworks using common deployment scripts.
*Can deploy clusters concurrently over multiple clouds.
*Deployments can be configured according to custom parameters (ex. number of nodes, services to install on the VM, etc...)
*VM contextualization is performed by Slipstream orchestrator (a separately instantiated VM), using SSH or native RP contextualization methods.
*Autoscaling capabilities will be implemented in 2014. Autoscaling will act at infrastructure level, using CPU/Disk/Memory metrics (not application metrix, ex. number of jobs in queue).
*RPs don't need to configure new services. It uses native cloud APIs.
*Provides EC2 (currently in beta) and SlipStream native REST API interface.
*User do not need to have cloud credentials to instantiate VMs. Slipstream acts as account broker.
*Does not provide accounting or cost information.
*Test interface available on the FedCloud for users, to request access please contact [mailto:salvatore.pinto@egi.eu salvatore.pinto@egi.eu]
 
|}
 
=== Application Brokers  ===
 
{| cellspacing="0" cellpadding="5" width="90%" class="wikitable" style="border:1px solid black;"
|- style="background-color:darkgray;"
! Name
! URL
! Cloud SW Support
! OCCI
! External API
! Functionalities
! Effort required
! Comments
|-
| COMPSs
| http://www.bsc.es/compss
| &nbsp; OpenNebula, Amazon, Azure, OpenStack
| Yes, support of rOCCI servers and OCCI+OVF
| OGF-BES SOAP
| &nbsp;Automatic parallelization and orchestration of applications and services, elasticity, auto scaling
| <span style="color:orange"> '''Medium'''</span>
|
*COMPSs allows the execution of applications on distributed infrastructures (grids, clusters, cloud).
*Already used for the BioVeL/OpenBio use case.
*Transparent support to different infrastructures; the user only provides his credentials.
*Automatic selection of the VMs types dependending on the tasks constraints.
*The number of VMs is dynamically adjusted to the current workload.
 
|-
| VMDIRAC
| https://github.com/DIRACGrid/VMDIRAC/wiki
| &nbsp; OpenNebula, OpenStack, CloudStack and Amazon EC2
| Yes, it supports rOCCI, OpenNebula "OCCI 0.8", Amazon EC2, OpenStack Nova 1.1 and CloudStack 2 APIs.
|
| Accounting, monitoring, brokering, scheduling, HEPiX and SSH based VM contextualisation, [https://github.com/DIRACGrid/VMDIRAC/wiki#vm-horizontal-auto-scaling-setup VM Horizontal Auto-scaling].
| <span style="color:orange"> '''Medium'''</span>
|
*rOCCI support with X509/VOMS auth.
*In production: LHCb (OpenStack and OpenNebula) Belle (Amazon), Work in progress France Federated Cloud.
*Web UI including VM browsing with VM monitoring, and VM Overview Plots (Load, RunningVM, RunnigVM by Endpoint, started jobs, data transfer, transfer files)
*The number of VMs is dynamically adjusted to the current workload.
*Automatic selection of the VMs types depending on the tasks constraints.
*No necessary to install any pluging in the Cloud Managers, works with standard interfaces.
*Separated user credentials are used for job running and VM operator credentials for VM management.
*VMDIRAC developers are willing to collaborate actively with Fedcloud Taskforce.
 
|}

Latest revision as of 12:11, 8 June 2015