Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Fedcloud-tf:WorkGroups:Scenario1"

From EGIWiki
Jump to navigation Jump to search
(Clean-up & new page structure)
Line 1: Line 1:
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}  
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}  


= Scenario 1: Running a pre-defined VM Image  =
<font color="green" size="5">This page is being updated. Displayed information might be incomplete or inconsistent. Please, come back later.</font>


<font color="red">Leader: Boris Parak, CESNET; Michel Drescher, EGI; Matteo Turilli, Oxford e-Research Centre</font>
= Scenario 1: Virtual Machine Management (OCCI) =


== Issues to follow-up/resolve  ==
<font color="red">Leader(s): Boris Parak, CESNET; Michel Drescher, EGI; David Wallom, Oxford e-Research Centre</font>


Following need to be considered with this scenario
== Available Solutions ==


#Trust level and Auditing of the VM (since it has to run as Root access)  
=== Open Standards ===
#Different VMs needed based on underlying Infrastructure such as 64 vs 32bits Or VT enabled plus Xen vs KVM
==== Open Cloud Computing Interface (OCCI) ====
#Contextualization i.e. how users should login to this vm , how his public key transfer and active to login as root to this vm
==== Cloud Infrastructure Management Interface (CIMI) ====
#Which libraries/versions/compilers will be installed by default?


From a user perspective (WeNMR contribution) we would like to:
=== De-facto Standards ===
==== AWS EC2 ====
==== OpenStack APIs ====


#Be able to install software on the pre-defined VM images (under the user account)  
=== Abstraction Libraries ===
#Be able to save those images (at least for a pre-defined time) (i.e. no new installation each time we wish to use the image)
==== Fog (Ruby) ====
#Be able to access persistent local storage from the VMs (Peachnote contribution)
==== jclouds (Java) ====
<br>
==== libcloud (Python) ====


'''''Note:'''''<i>This section is heavily in-flux so the information given above might be slightly outdated. The minutes of the conference calls contain more information.</i>
== Open Cloud Computing Interface (OCCI) ==
=== Examples ===
=== Implementations ===
=== Deployment ===


== OCCI interface testing  ==
== FAQ ==
 
The Task Force decided to use the OCCI interface as the canonical, task force-wide uniform interface for API-based VM Management. To ensure this uniformity&nbsp;(and applicability of the OCCI interface) a series of test cases are published and maintained that aim to cover all necessary funtionality to manage Virtual Machines.
 
Starting with two initial volunteer sites, the matrix testing effort explicitly aims at expanding out to Resource Providers that run other Cloud Management solutions such as:
 
*OpenStack (FZJ, OerC, etc),
*JClouds based providers (e.g. CloudSigma),
*proprietary solutions (GRNET)
 
once OCCI implementations become available for the respective solution in use.
 
=== OCCI Implementations  ===
 
{| cellspacing="0" cellpadding="3" border="1" style="border-collapse: collapse; border:1px solid black; text-align:center;" class="wikitable"
|+ '''OCCI Implementations for OpenStack'''
|-
| Project
| Contact
| URL
| Status
| Comment
|-
| FI-WARE (INTEL)
| [mailto:andy.edmonds@gmail.com Andrew Edmonds]
| http://www.fi-ppp.eu/projects/fi-ware/
| Availability targeted for February
|
*Implemented in python
*Positioned as official OCCI-Implementation<br>for OpenStack
 
|-
| CompatibleOne
| [mailto:ijm667@hotmail.com Iain James Marshall]
| http://www.compatibleone.fr/occi/publisher/occiserver.htm
| released
| Implemented in C
|}
 
<br>
 
{| cellspacing="0" cellpadding="3" border="1" style="border-collapse: collapse; border:1px solid black; text-align:center;" class="wikitable"
|+ '''OCCI Implementations for jclouds'''
|-
| Project
| Contact
| URL
| Status
| Comment
|-
| ...
| ...
| ...
| ...
| ...
|}
 
<br>
 
<br>
 
=== Test VM Image  ===
 
The reference VM is available at: http://repository.egi.eu/mirrors/vms/egi/
 
=== Status &amp; Outcomes  ===
 
The following is organised as some sort of dashboard showing the progress and status of the testing. The second table shows the progress in preparation for participating Resource Providers on how far they are until testing may commence (either by them, or against their OCCI implementation).
<div style="width:100%;height:100%;overflow:hidden;"><div style="width:33%;float:left;">
'''Matrix testing progress'''
 
{| cellspacing="0" cellpadding="3" border="1" style="border-collapse: collapse; border:1px solid black; text-align:center;" class="wikitable"
|- style="background-color: lightgray;"
! rowspan="2" | Client
! colspan="6" | OCCI Services
|- style="background-color: lightgray;"
| CESGA
| GWDG
| CESNET
| GRNET
| IGI
| KTH
|-
| EGI.eu
|
|
|
|
|
|
|-
| CESGA
| style="background:lightgray" |
|
|
|
|
|
|-
| GWDG
|
| style="background:lightgray" |
|
|
|
|
|-
| CESNET
| style="background:yellow" |
| style="background:yellow" |
| style="background:lightgray" |
|
|
|
|-
| GRNET
|
|
|
| style="background:lightgray" |
|
|
|-
| IGI
|
|
|
|
| style="background:lightgray" |
|
|-
| KTH
|
|
|
|
|
| style="background:lightgray" |
|}
 
Legend:
 
{| cellspacing="0" cellpadding="0" border="0"
|-
| style="background:green" | &nbsp;&nbsp;&nbsp;
| &nbsp; = All tests successful
|-
| style="background:yellow" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Testing in progress
|-
| style="background:red" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Testing finished, but with errors.
|-
| style="background:lightgray" | &nbsp;&nbsp;&nbsp;
| &nbsp; = not applicable
|-
| style="background:white; border:1px solid black;" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Task not started
|}
</div> <div style="width:33%;float:left;">
'''Resource Provider preparation status:'''
 
{| cellspacing="0" cellpadding="3" border="1" style="border-collapse: collapse; border:1px solid black; text-align:center;" class="wikitable"
|- style="background-color: lightgray;"
! rowspan="2" | Preparation step
! colspan="7" | Participant
|- style="background-color: lightgray;"
| <span class="plainlinks">[[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:EGIeu|EGI.eu]]</span>
| <span class="plainlinks">[[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESGA|CESGA]]</span>
| GWDG
| <span class="plainlinks">[[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESNET|CESNET]]</span>
| <span class="plainlinks">[[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:GRNET|GRNET]]</span>
| <span class="plainlinks">[[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:IGI|IGI]]</span>
| <span class="plainlinks">[[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:KTH|KTH]]</span>
|-
| Create a test account
| style="background:yellow" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:EGIeu#Create_a_test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESGA#Test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:GWDG#Test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESNET#Test_account]
|
|
|
|-
| Provision the test VM
| n/a
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESGA#Test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:GWDG#Test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESNET#Test_VM]
|
|
| 
|-
| OCCI test client provisioning
|
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESGA#Test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:GWDG#Test_account]
| style="background:green" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESNET#OCCI_test_client]
|
|
|
|-
| Run tests
|
|
|
| style="background:yellow" | [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CESNET#Testing]
|
|
|}
 
Legend:
 
{| cellspacing="0" cellpadding="0" border="0"
|-
| style="background:green" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Action completed.
|-
| style="background:yellow" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Action in progress
|-
| style="background:red" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Stuck, can't proceed. Requires coordination.
|-
| style="background:white; border:1px solid black;" | &nbsp;&nbsp;&nbsp;
| &nbsp; = Task not started
|}
</div></div>
 
=== Execution plan  ===
 
The following is a list of actions that need to be accomplished before the Matrix testing can start:
 
*'''Establish a [[Fedcloud-tf:WorkGroups:Scenario1:OCCITesting:CanonicalVM|canonical test VM]] with an EchoService, featuring:'''
**Allows a user to ssh into the machine, and query VM properties (such RAM, CPUs, etc),
**The EchoService starts when the VM boots up (e.g. via Linux init-scripts)
***The service listens at a predefined port
***A user (may be hard coded) telnets into that socket
***Whichever text message the user sends will get returned (echoed) with a little server prefix (e.g. current time &amp; server IP)
***If the user sends "EXIT" then the connection to the echo service is terminated
 
Each site needs to prepare as follows:
 
*'''Create a test account'''<br>
**For now username &amp; password. Perhaps individual users for each other participating Resource Centre.
**That user need not be the same that connects to the echo service (see above)
*'''Provision the test VM'''
**Provisions the reference VM for the test user.
***The reference VM is available at http://repository.egi.eu/mirrors/vms/egi/.
***A running reverence version is available at test29.egi.cesga.es:8000 for telnet access.
**For minimum security the test user should ''only'' have access to that test VM and nothing else.
*'''OCCI test client provisioning'''
**Each Resource Provider should, ideally, use an OCCI client on their own for testing against other participants.
*'''Run tests'''
**Run the tests defined in section [[Fedcloud-tf:WorkGroups:Scenario1#Test_cases|Test cases]] against each participating Resource Provider.
**Document the status and outcome of the tests [[Fedcloud-tf:WorkGroups:Scenario1#Status_.26_Outcomes|Status &amp; Outcomes]].
 
=== Test cases  ===
 
This section describes a set of agreed test cases (see below) and a set of testcases that need further discussion whether to include them or not.
 
==== Test cases that need discussion  ====
 
#'''Change VM attributes while the VM is active.'''<br>I haven't found any specific information about this behaviour. I think it is useful to test, but this might go too foar for the purpose of the task force. <br> I don't really follow what this could mean. Which attributes are we discussing here? Josef Pacula
 
==== Agreed test cases  ====
 
The following test cases have been identified as the minimal set of coverage testing the OCCI capabilities:
 
#'''Identify and query the VM'''
##Query for a listing of compute resources.
###Fail if the pre-provisioned VM is not in the list.
##Query the pre-provisioned compute resource.
###Fail if it cannot be queried for detailed information.
#'''Change the VM attributes'''
##Query for the VM.
###Fail if the instance is NOT in state 'inactive'
##Change the attribute ''occi.compute.cores'' to a value different than given in the query.
##Change the attribute ''occi.compute.hostname'' to a value different than given in the query.
##Change the attribute ''occi.compute.speed'' to a value different than given in the query.
##Change the attribute ''occi.compute.memory'' to a value different than given in the query.
##Query again for the compute resource.
###Fail if at least one attribute <u>did not</u> change its value.
#'''Start the VM'''
##Query for the VM.
###Fail if it is not listed, or NOT in state ''inactive''.
##Invoke the action ''start'' on the VM.
##Monitor the VM state by querying for the VM AND trying to connect to the VM (ping, or using the Echo service).
###Fail if the VM does not start up (i.e. does not arrive at state ''active'').
##Connect to the Echo service and send some messages around.
###Fail if the echo service cannot be contacted.
#'''Restart the VM'''
##<u>Prerequisite:</u> Test 3 succeeded (i.e. VM in state ''active'').
##Invoke action ''restart'' on the VM, using the ''warm'' method.
##Monitor the VM state by querying for the VM AND trying to connect to the VM (ping, or using the Echo service).
###Fail if the VM does not start up
##The retrieved VM state must transition, in this order, through the states ''active'' --&gt; ''inactive'' --&gt; ''active''
##Connect to the Echo service and send some messages around.
###Fail if the echo service cannot be contacted.
#'''Suspend/resume the VM'''
##<u>Prerequisite:</u> Test 3, or Test 4 succeeded (i.e. VM in state ''active'').
##Invoke action ''suspend'' on the VM, using the ''hibernate'' method.
##Monitor the VM state by querying for the VM.
###Fail if the VM does transition into state ''suspended''.
##Connect to the Echo service and send some messages around.
###Fail if the echo service can be contacted to echo messages back.
##Invoke action ''start'' on the VM.
##Monitor the VM state by querying for the VM AND trying to connect to the VM (ping, or using the Echo service).
###Fail if the VM does not transition into state ''active''.
##Connect to the Echo service and send some messages around.
###Fail if the echo service cannot be contacted.
#'''Stop the VM'''
##<u>Prerequisite:</u> Test 3, 4 or Test 5 succeeded (i.e. VM in state ''active'').
##Query for the VM.
###Fail if it is not listed, or NOT in state ''active''.
##Connect to the Echo service and send some messages around.
##Invoke the action ''stop'' on the VM.
##Monitor the VM state by querying for the VM AND trying to connect to the VM (ping, or using the Echo service).
###Fail if the VM does not assume the state ''inactive''.


== Collaborators  ==
== Collaborators  ==
Line 351: Line 37:
! Name
! Name
|-
|-
| Scenario leader  
| Scenario leader(s)
| CESNET, EGI.eu, OeRC  
| CESNET, EGI.eu, OeRC  
| Boris Parak, Michel Drescher, Matteo Turilli
| Boris Parak, Michel Drescher, David Wallom
|-
| Collaborator
| CESNET
| Zdeněk Šustr
|-
|-
| Collaborator  
| Collaborator  
| GWDG  
| GWDG  
| Piotr Kasprzak
| Piotr Kasprzak
|-
| Collaborator
|
|
|}
|}

Revision as of 15:42, 7 August 2014

Main Roadmap and Innovation Technology For Users For Resource Providers Media


Workbenches: Open issues
 		Scenario 1
VM Management 		Scenario 2
Data Management 		Scenario 3
Information Systems 		Scenario 4
Accounting 		Scenario 5
Monitoring 		Scenario 6
Notification 		Scenario 7
Federated AAI 		Scenario 8
VM Image Management 		Scenario 9
Brokering 		Scenario 10
Contextualisation 		Scenario 11
Security



This page is being updated. Displayed information might be incomplete or inconsistent. Please, come back later.

Scenario 1: Virtual Machine Management (OCCI)

Leader(s): Boris Parak, CESNET; Michel Drescher, EGI; David Wallom, Oxford e-Research Centre

Available Solutions

Open Standards

Open Cloud Computing Interface (OCCI)

Cloud Infrastructure Management Interface (CIMI)

De-facto Standards

AWS EC2

OpenStack APIs

Abstraction Libraries

Fog (Ruby)

jclouds (Java)

libcloud (Python)

Open Cloud Computing Interface (OCCI)

Examples

Implementations

Deployment

FAQ

Collaborators

Role Institution Name
Scenario leader(s) CESNET, EGI.eu, OeRC Boris Parak, Michel Drescher, David Wallom
Collaborator CESNET Zdeněk Šustr
Collaborator GWDG Piotr Kasprzak
Retrieved from "https://wiki.egi.eu/w/index.php?title=Fedcloud-tf:WorkGroups:Scenario1&oldid=69280"