Difference between revisions of "Fedcloud-tf:WorkGroups:Contextualisation"
Line 99: | Line 99: | ||
There are OCCI-OS and rOCCI server implementations ready to support the <pre>Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"</pre>mixin with the <pre>org.openstack.compute.user_data</pre> attribute. These versions are being gradually deployed at the testbed. | There are OCCI-OS and rOCCI server implementations ready to support the <pre>Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"</pre>mixin with the <pre>org.openstack.compute.user_data</pre> attribute. These versions are being gradually deployed at the testbed. | ||
== | == Contextualization at the VMs == | ||
The proposed mixin does not impose any format on the data, thus the VM image creators are free to use this blob as it suites their needs. This task force recommends the use of [https://help.ubuntu.com/community/CloudInit|cloud-init] for handling the data. Cloud-init is supported in debian and redhat based distributions and there is some work in progress to make it aware of the OpenNebula contextualization mechanisms. | The proposed mixin does not impose any format on the data, thus the VM image creators are free to use this blob as it suites their needs. This task force recommends the use of [https://help.ubuntu.com/community/CloudInit|cloud-init] for handling the data. Cloud-init is supported in debian and redhat based distributions and there is some work in progress to make it aware of the OpenNebula contextualization mechanisms. |
Revision as of 10:26, 15 September 2013
Main | Roadmap and Innovation | Technology | For Users | For Resource Providers | Media |
Leader: Enol Fernandez
Scope
This workgroup deals with the VM contextualization open issues, namely being able to pass user-defined data to the VM.
Roadmap
- Evaluate mechanisms available in the cloud middleware to provide user-defined data
- Evaluate possible OCCI extensions to pass user-defined data
- Evaluate CIMI as alternative API (already includes user-data features)
- Track implementation of OCCI extensions/CIMI on the cloud middleware
Cloud middleware support
OpenStack
OpenStack provides a meta-data server that provides instance-specific data. Instances can access the metadata service at http://169.254.169.254 using two different interfaces: one EC2-compatible and a proprietary Openstack interface.
The meta-data server provides the user public ssh-keys if included in the VM request and a user_data field that can be used freely by the user. cloud-init uses this field to contextualize machines.
OpenStack also allows file injection into the VM instance. These files can also be used for passing user-defined information to the running instance.
OpenNebula
OpenNebula provides contextualization information through an ISO image that is available in the VM instance. The contents of the ISO image include a context.sh file where a set of variables are defined. The contents of the context.sh are defined in the VM template.
A EC2-compatible meta-data server is also available at https://bitbucket.org/ricardoduarte/opennebula-metadata. It needs to be tested.
Stratuslab
StratusLab supports CloudInit, HEPiX and OpenNebula contextualization.
For HEPiX and OpenNebula contextualization, an ISO CDROM is attached to the VM that supports the file layout for both systems.
CloudInit contextualization handles web-server and disk based contextualization. StratusLab supports the second one. CloudInit only supports disks formatted as VFAT file systems at the moment. More information can be found at CloudInit in StratusLab documentation
Currently, the OpenNebula contextualization is the default. In the upcoming releases, the default will become CloudInit. All three methods for contextualization will be supported for the foreseeable future.
Okeanos/Synefo
Okeanos uses OpenStack API, in particular it supports the file-injection via http://docs.openstack.org/api/openstack-compute/2/content/Server_Personality-d1e2543.html
OCCI extension
OCCI does not include support for passing user-provided data to the instances.
We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined:
- how to pass data upon VM creation
- how to retrieve those data from the running VM
For passing the data we propose the use of a new mixin that has an attribute to hold the data to pass to the image. We have a working modification of the OCCI-OS implementation that uses such mixin at https://github.com/enolfc/occi-os/tree/occi_user_data. In this case we have defined a mixin:
Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"
and
org.openstack.compute.user_data
as attribute (base64 encoded), that uses the user_data field of the OpenStack API.
We think a similar mixin could be added to OCCI to allow interoperability between resource providers.
The second part, related to retrieving the data, is more dependent of the back-end implementation. We have identified 3 different methods in the systems in place in FedCloud: - a metadata server (a la EC2), that located in a known URI servers the user-data - attach of an iso image to the VM that can be mounted - injection of a file with the user-data
All of those methods should be allowed and the VM once created should try them in order (the least invasive seems the metadata server, in our tests the file injection may fail with some VM images)
We think that the exact type and format of the data is not relevant. There could be specialized mixins to indicate explicitly that the data conforms to a executable script or to Cloud-init, but free-format data should also be allowed (for example at IFCA we have developed a system to contextualize just passing a JSON dictionary)
CIMI
We have deployed an instance of OpenStack for testing the CIMI implementation from this blueprint. A simple modification adds the support for the `userData` parameter in machine creation.
Sample code for the request (python code with requests package):
headers = {'content-type': 'application/json', 'accept': 'application/json', 'X-Auth-Token': keystone_token} img_body = { 'name': 'cimi-test-server', 'description': 'no desc', 'machineTemplate': { 'machineImage': {'href': image_id}, 'machineConfig': {'href': conf_id}, 'userData': base64.b64encode('this is just a test'), } } r = requests.post(cimi_url + '/machineCollection', headers=headers, data=json.dumps(img_body))
Implementation status
There are OCCI-OS and rOCCI server implementations ready to support the
Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"
mixin with the
org.openstack.compute.user_data
attribute. These versions are being gradually deployed at the testbed.
Contextualization at the VMs
The proposed mixin does not impose any format on the data, thus the VM image creators are free to use this blob as it suites their needs. This task force recommends the use of [1] for handling the data. Cloud-init is supported in debian and redhat based distributions and there is some work in progress to make it aware of the OpenNebula contextualization mechanisms.