|
|
| Line 1: |
Line 1: |
| {{Fedcloud_Menu}} | | {{Fedcloud_Menu}} |
| | {{FedCloud_Innovation_Menu}} |
|
| |
|
| {{FedCloud_TF_Menu}} {{TOC_right}}
| | {{TOC_right}} |
|
| |
|
| <br>
| | = AAI = |
|
| |
|
| == Mandate ==
| | EGI is transitioning to a new [[AAI|AAI services]] that provides Single Sign-On (SSO) using federated authentication mechanism. FedCloud is working to support SAML and OpenID Connect at the resource centres to support certificate-less access for the users to these resources. First test with OpenStack are already working and first production sites are expected during Q4 2016. |
|
| |
|
| EGI is a federation of national and domain specific resource infrastructure providers comprised of individual resource centres. Many of these resource centres have been experimenting with the deployment of virtualised management environments to improve the local delivery of services. Many of EGI’s current and new user communities would like to access the flexibility provided by virtualisation across the infrastructure on demand in a ‘cloud like’ environment. Federating these individual virtualised resources is a major priority for EGI that has started with the [http://go.egi.eu/uvw1 EGI User Virtualisation Workshop], and the drafting of the [http://go.egi.eu/435 EGI Cloud Integration Profile].
| | = VM Management = |
|
| |
|
| Objectives:
| | == OCCI 1.2 == |
|
| |
|
| *write a [[Federated_Cloud_Blueprint|blueprint document]] for EGI Resource Providers that wish to [[Federated_Cloud_Federated_AAI|securely federate]] and share their virtualised environments as part of the EGI production infrastructure;
| | The Open Cloud Computing Interface (OCCI) is a RESTful Protocol and API for all kinds of Management tasks. OCCI was originally initiated to create a remote management API for IaaS model based Services, allowing for the development of interoperable tools for common tasks including deployment, autonomic scaling and monitoring. It has since evolved into an flexible API with a strong focus on interoperability while still offering a high degree of extensibility. |
| *deploy a [[Federated_Cloud_Operation|test bed]] to evaluate the integration of virtualised resources within the existing EGI production infrastructure for [[Federated_Cloud_Monitoring|monitoring]], [[Federated_Cloud_Accounting|accounting]] and [[Federated_Cloud_Information_Discovery|information services]];
| |
| *investigate and catalogue the requirements for community facing services based on or deployed through virtualised resources;
| |
| *provide feedback to relevant technology providers on their implementations and any changes needed for deployment into the production infrastructure;
| |
| *identify and work with [[Federated Cloud user support|user communities]] willing to be early adopters of the test bed infrastructure to help prioritise its future development;
| |
| *identify issues that need to be addressed by other areas of EGI (e.g. policy, operations, support & dissemination).
| |
|
| |
|
| == Activities ==
| | The next revision of OCCI will be 1.2, due in Q3 2016. Specification is already available at http://occi-wg.org/about/specification/. EGI Federated Cloud will implement the new specification soon after its official release. |
|
| |
|
| The Task Force mandate lasts eighteen months, from Sept 2011 to March 2013. The Task Force activities are organised in three, six-months long phases. During each phase, the Task Force evaluates a set of [[Fedcloud-tf:WorkGroups|scenarios]] that an EGI federation of clouds should support. The scenarios are chosen by collecting use cases and requirements among user communities, resource providers and technology providers that have already adopted cloud computing or are planning to do so in a near future.
| | == OCCI for public commercial clouds == |
|
| |
|
| One or more [[Fedcloud-tf:WorkGroups|work group]] is created inside the Task Force in order to evaluate each scenario. A leader is appointed for each work group and one or more collaborators are chosen among the Task Force members. The scenario evaluation performed by each work group is recorded in [[Fedcloud-tf:WorkGroups|workbenches]] and consists into:
| | rOCCI server architecture allows different backends to be plugged to support new cloud management frameworks. The rOCCI team has started to implement backends for Amazon AWS and Microsoft Azure. |
|
| |
|
| *defining the set of capabilities that an EGI cloud infrastructure should have in order to support the given scenario;
| | == VM Operations Dashboard == |
| *evaluating whether and what standards are available to implement the required capabilities;
| |
| *evaluating whether and what software solution is available to implement the required standards;
| |
| *evaluating the level of support of such software solutions by the Resource Providers members of the Task Force;
| |
| *evaluating the procedures to deploy the required software solutions into the infrastructure of the Resource Providers;
| |
| *devising tests to be run on the cloud infrastructure of the Resource Providers in order to simulate the given scenario.
| |
|
| |
|
| Once the scenario has been evaluated and, when possible, the required software and tests have been deployed in the Task Force [[Fedcloud-tf:Testbed|test bed]] federated cloud, the members of the work group proceed to write a section of the [[Fedcloud-tf:Blueprint|blueprint]] document. Altogether, the blueprint document describes the set of capabilities that define an EGI Cloud Federation. The blueprint focuses on the infrastructure layer of the federation and each work group is tasked with documenting at least one core capability. Each section of the blueprint is structured as follow.
| | AppDB will evolve from its current role as catalogue of applications and virtual machines to also include a graphical user interface allowing authorised users to perform basic VM management operations. This will allow users to interact with the Federated Cloud without command line interface experience. First version of the dashboard will be available Q3 2016. |
|
| |
|
| *'''Overview'''. The scenario and use cases from which the capability has been investigated are briefly introduced.
| |
| *'''Definition of the capability'''. The functional properties, interfaces and available standards for implementation are described and analysed. A reasoned choice is made about what standards and implementations should be endorsed by an EGI cloud federation.
| |
| *'''Deployment scenarios'''. The deployment infrastructure(s) and topology(ies) for the implementation of the capabilities are described and analysed.
| |
| *'''Testing the capability'''. When possible, a description is given of the tests used to simulate the scenario requiring the capability under exam.
| |
| *'''Security considerations'''. Security is a particularly sensitive topic in a federated infrastructure. Each capability has its own security evaluation with references to the EGI policy documents and security guidelines.
| |
| *'''Solution inventory'''. A critical review of how the capability under exams is implemented – or not implemented - by the Virtual Machine Management platforms deployed by the Resource Providers participating to the Task Force.
| |
|
| |
|
| The progress of the activity of each work group is available in the [[Fedcloud-tf:WorkGroups:Workbenches|workbenches summary]] section.
| | = Information Discovery = |
|
| |
|
| == Operations ==
| | Current information published about the resources in the FedCloud is not enough to perform automated discovery for higher level tools. A new schema based on Glue Schema 2.1 is being developed and implementations are to be available in Q3 2016 |
|
| |
|
| The Task Force operates on its [mailto:fedcloud-tf@mailman.egi.eu mailing list] and on a [https://indico.egi.eu/indico/categoryDisplay.py?categId=159 weekly conference call]. At the moment, our effort is organised in [[Federated_Cloud_Task_Force#FedCloud_work_groups|work groups]] that any member of the Task Force is free to lead or to contribute to. Each member of Task Force can organise a meeting dedicated to further progress a specific workgroup activity.
| | = VM Image Management = |
|
| |
|
| The Task Force meet every Tuesday at 11:00 CET in a [https://indico.egi.eu/indico/categoryDisplay.py?categId=159 conference call]. Minutes of the meetins are publicly available in the INDICO pages. <!--
| | A complete rewrite of the VM Image replication components is undergoing, this will substitute vmcatcher currently in use in the federation which has some reliability problems. The new implementation will avoid inconsistencies by relying on the cloud management framework image catalogues instead of keeping local databases. This new component will be pluggable to allow using different cloud management framework technologies easily. |
| <div style="width:100%; height:100%; overflow:hidden;>
| |
| <div style="width:56%; float:left; text-align:center;">
| |
| High attendance
| |
| <pBars ymin='0' ymax='100' angle='60' yunit='%' size='510x400' titlecolor='#000000' boxcolor='#000000' axiscolor='#000000' colors='#00AF33' marginx='15'>
| |
| OeRC (UK NGI),100
| |
| Chair,100
| |
| EGI-InSPIRE SA2,100
| |
| STFC (UK NGI),100
| |
| STFC (e-Science),100
| |
| EGI.eu,100
| |
| Operations Officer,92
| |
| TCD (NGI IE),90
| |
| Technology Manager,84
| |
| CESGA (IBERgrid),80
| |
| Community Officer,76
| |
| IGI,75
| |
| CYFRONET (NGI PL),69
| |
| CloudSigma,66
| |
| INFN/CNAF,66
| |
| EGI-InSPIRE JRA1,66
| |
| FZ Julich,66
| |
| </pBars>
| |
| </div>
| |
| <div style="width:23%; float:left; text-align:center;">
| |
| Medium attendance
| |
| <pBars ymin='0' ymax='100' angle='60' yunit='%' size='210x400' titlecolor='#000000' boxcolor='#000000' axiscolor='#000000' colors='#FCD116' marginx='15'>
| |
| CESNET (NGI CZ),61
| |
| DANTE,57
| |
| GWDG,54
| |
| GRNET,50
| |
| WeNMR,45
| |
| KTH,41
| |
| IN2P3 (NGI FR),40
| |
| </pBars>
| |
| </div>
| |
| <div style="width:20%; float:left; text-align:center;">
| |
| Low attendance
| |
| <pBars ymin='0' ymax='100' angle='60' yunit='%' size='180x385' titlecolor='#000000' boxcolor='#000000' axiscolor='#000000' colors='#FF0000' marginx='15'>
| |
| SARA (NGI NL),30
| |
| StratusLab,23
| |
| OGF DCIFed,18
| |
| SIENA Project,9
| |
| IPHC,0
| |
| GRIF,0
| |
| </pBars>
| |
| <br>
| |
| </div>
| |
| </div>
| |
|
| |
|
| Full details on meeting attendance are published in the [[Fedcloud-tf:Administrativa#Meeting_attendance| administrative]] section
| | = Accounting = |
| --><br>
| |
|
| |
|
| = FedCloud work groups =
| | New format for the accounting records (v0.4) is now available for deploying at the sites. This new version improves the data collected at the sites. |
| | Accounting of long running VMs is being implemented, this will allow to have information on the accounting portal about VMs that are not yet terminated. |
|
| |
|
| The FedCloud Task Force activities are split across work groups. A leader is elected for each work group and members of the Task Force are free to spend their effort in one or more groups. Each work group investigates one or more capabilities that are required by a federation of clouds. The work done is recorded in the group workbench and, eventually, translated into the Task Force blueprint.
| | = Monitoring = |
|
| |
|
| With the development of the testbed and of the blueprint, new capabilities will be investigated and addressed. As a consequence, new work groups are added to the Task Force when required.
| | Monitoring probes for detecting common issues and new services are planned: |
| | * Probe for OpenStack Swift interface |
| | * Probe for detecting inconsistencies on the images available at the site |
|
| |
|
| == Running == | | = Networking = |
|
| |
|
| *[[Federated_Cloud_VM_Management |VM Management]] - leader: Boris Parák (CESNET)
| | The networking group is designing new OCCI extensions to deal with public IPs, security groups and private networks taking into consideration the heterogeneity of the providers. |
| *[[Federated_Cloud_Data_Management| Data Management]] - leader: <span><span>Christos</span> <span>Loverdos</span> (GRNET)</span>
| | |
| *[[Federated_Cloud_Information_Discovery| Information Discovery]] - leader:<span><span> Peter</span> <span>Solagna</span> (EGI.EU)</span>
| |
| *[[Federated_Cloud_Accounting| Accounting]] - leader: <span><span>Stuart</span> <span>Pullinger</span> (STFC)</span>
| |
| *[[Federated_Cloud_Monitoring| Monitoring]] - leader: <span><span>Emir</span> <span>Imamagic</span> (SRCE)</span>
| |
| *[[Federated_Cloud_Federated_AAI| federated AAI]] - leader: <span> <span>Paul</span> <span>Millar</span> (DESY)</span>
| |
| *[[Federated_Cloud_VM_Image_Management| VM Image Management]] - leader:<span><span> Marios</span> <span>Chatziangelou</span> (IASA)</span>
| |
| *[[Federated_Cloud_Brokering| Brokering]] - leader: (interim) Enol Fernandez (EGI.eu) <br>
| |
| *[[Federated_Cloud_Security| Security]] - leader:Linda Cornwall (STFC)
| |
| *[[Federated_Cloud_IntraCloud_Networking|Intra Cloud networking]] - leader: Zdenek Sustr(CESNET)
| |
|
| |
|
| == Closed ==
| |
|
| |
| *[https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario6 Notification]
| |
| *[https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Contextualisation Contextualisation]
| |
|
| |
| = Join the EGI FedCloud Task Force =
| |
|
| |
| If you are interested in joining the EGI FedCloud Task Force, please send a mail to [mailto:fedcloud-tf@mailman.egi.eu fedcloud Mailing list] introducing yourself.
| |
|
| |
| = Resources =
| |
|
| |
| #[mailto:fedcloud-tf@mailman.egi.eu Mailing list] (incl. [https://mailman.egi.eu/mailman/private/fedcloud-tf/ Mailing list archive])
| |
| #[https://indico.egi.eu/indico/categoryDisplay.py?categId=159 Meeting planner]
| |
|
| |
| = References =
| |
|
| |
| #'''Task Force Mandate''': [https://documents.egi.eu/document/720 https://documents.egi.eu/document/720]
| |
| #'''Task Force Roadmap''': [https://documents.egi.eu/secure/RetrieveFile?docid=803&version=1&filename=lyon_20092011_v3.pdf https://documents.egi.eu/secure/RetrieveFile?docid=803&version=1&filename=lyon_20092011_v3.pdf]
| |
| #'''Other EGI Virtualisation activities''': https://wiki.egi.eu/wiki/EGI_Virtualisation_Activities
| |
|
| |
| <br>
| |
|
| |
|
| [[Category:Federated_Cloud]] | | [[Category:Federated_Cloud]] |
AAI
EGI is transitioning to a new AAI services that provides Single Sign-On (SSO) using federated authentication mechanism. FedCloud is working to support SAML and OpenID Connect at the resource centres to support certificate-less access for the users to these resources. First test with OpenStack are already working and first production sites are expected during Q4 2016.
VM Management
OCCI 1.2
The Open Cloud Computing Interface (OCCI) is a RESTful Protocol and API for all kinds of Management tasks. OCCI was originally initiated to create a remote management API for IaaS model based Services, allowing for the development of interoperable tools for common tasks including deployment, autonomic scaling and monitoring. It has since evolved into an flexible API with a strong focus on interoperability while still offering a high degree of extensibility.
The next revision of OCCI will be 1.2, due in Q3 2016. Specification is already available at http://occi-wg.org/about/specification/. EGI Federated Cloud will implement the new specification soon after its official release.
OCCI for public commercial clouds
rOCCI server architecture allows different backends to be plugged to support new cloud management frameworks. The rOCCI team has started to implement backends for Amazon AWS and Microsoft Azure.
VM Operations Dashboard
AppDB will evolve from its current role as catalogue of applications and virtual machines to also include a graphical user interface allowing authorised users to perform basic VM management operations. This will allow users to interact with the Federated Cloud without command line interface experience. First version of the dashboard will be available Q3 2016.
Information Discovery
Current information published about the resources in the FedCloud is not enough to perform automated discovery for higher level tools. A new schema based on Glue Schema 2.1 is being developed and implementations are to be available in Q3 2016
VM Image Management
A complete rewrite of the VM Image replication components is undergoing, this will substitute vmcatcher currently in use in the federation which has some reliability problems. The new implementation will avoid inconsistencies by relying on the cloud management framework image catalogues instead of keeping local databases. This new component will be pluggable to allow using different cloud management framework technologies easily.
Accounting
New format for the accounting records (v0.4) is now available for deploying at the sites. This new version improves the data collected at the sites.
Accounting of long running VMs is being implemented, this will allow to have information on the accounting portal about VMs that are not yet terminated.
Monitoring
Monitoring probes for detecting common issues and new services are planned:
- Probe for OpenStack Swift interface
- Probe for detecting inconsistencies on the images available at the site
Networking
The networking group is designing new OCCI extensions to deal with public IPs, security groups and private networks taking into consideration the heterogeneity of the providers.
The wiki is deprecated and due to be decommissioned by the end of September 2022.