Difference between revisions of "Fedcloud-tf:WorkGroups:Contextualisation"
Line 48: | Line 48: | ||
We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined: | We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined: | ||
* how to pass data upon VM creation | |||
* how to retrieve those data from the running VM | |||
For passing the data we propose the use of a new mixin that has an attribute to hold the data to pass to the image. We have a working modification of the OCCI-OS implementation that uses such mixin at https://github.com/enolfc/occi-os/tree/occi_user_data. In this case we have defined a mixin: | For passing the data we propose the use of a new mixin that has an attribute to hold the data to pass to the image. We have a working modification of the OCCI-OS implementation that uses such mixin at https://github.com/enolfc/occi-os/tree/occi_user_data. In this case we have defined a mixin: | ||
Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin" | <pre>Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"</pre> | ||
and org.openstack.compute.user_data as attribute (base64 encoded), that uses the user_data field of the OpenStack API. | and <pre>org.openstack.compute.user_data</pre> as attribute (base64 encoded), that uses the user_data field of the OpenStack API. | ||
We think a similar mixin could be added to OCCI to allow interoperability between resource providers. | We think a similar mixin could be added to OCCI to allow interoperability between resource providers. | ||
Line 62: | Line 62: | ||
- injection of a file with the user-data | - injection of a file with the user-data | ||
All of those methods should be allowed and the VM once created should try them in order (the least invasive seems the metadata server, in our tests the file injection may fail with some VM images) | |||
We think that the exact type and format of the data is not relevant. There could be specialized mixins to indicate explicitly that the data conforms to a executable script or to Cloud-init, but free-format data should also be allowed (for example at IFCA we have developed a [https://github.com/enolfc/feynapps system to contextualize] just passing a JSON dictionary) | We think that the exact type and format of the data is not relevant. There could be specialized mixins to indicate explicitly that the data conforms to a executable script or to Cloud-init, but free-format data should also be allowed (for example at IFCA we have developed a [https://github.com/enolfc/feynapps system to contextualize] just passing a JSON dictionary) |
Revision as of 11:48, 16 July 2013
Main | Roadmap and Innovation | Technology | For Users | For Resource Providers | Media |
Leader: Enol Fernandez
Scope
This workgroup deals with the VM contextualization open issues, namely being able to pass user-defined data to the VM.
Roadmap
- Evaluate mechanisms available in the cloud middleware to provide user-defined data
- Evaluate possible OCCI extensions to pass user-defined data
- Evaluate CIMI as alternative API (already includes user-data features)
- Track implementation of OCCI extensions/CIMI on the cloud middleware
Cloud middleware support
OpenStack
OpenStack provides a meta-data server that provides instance-specific data. Instances can access the metadata service at http://169.254.169.254 using two different interfaces: one EC2-compatible and a proprietary Openstack interface.
The meta-data server provides the user public ssh-keys if included in the VM request and a user_data field that can be used freely by the user. cloud-init uses this field to contextualize machines.
OpenStack also allows file injection into the VM instance. These files can also be used for passing user-defined information to the running instance.
OpenNebula
OpenNebula provides contextualization information through an ISO image that is available in the VM instance. The contents of the ISO image include a context.sh file where a set of variables are defined. The contents of the context.sh are defined in the VM template.
A EC2-compatible meta-data server is also available at https://bitbucket.org/ricardoduarte/opennebula-metadata. It needs to be tested.
Stratuslab
StratusLab supports CloudInit, HEPiX and OpenNebula contextualization.
For HEPiX and OpenNebula contextualization, an ISO CDROM is attached to the VM that supports the file layout for both systems.
CloudInit contextualization handles web-server and disk based contextualization. StratusLab supports the second one. CloudInit only supports disks formatted as VFAT file systems at the moment. More information can be found at CloudInit in StratusLab documentation
Currently, the OpenNebula contextualization is the default. In the upcoming releases, the default will become CloudInit. All three methods for contextualization will be supported for the foreseeable future.
Okeanos/Synefo
Okeanos uses OpenStack API, in particular it supports the file-injection via http://docs.openstack.org/api/openstack-compute/2/content/Server_Personality-d1e2543.html
OCCI extension
OCCI does not include support for passing user-provided data to the instances.
We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined:
- how to pass data upon VM creation
- how to retrieve those data from the running VM
For passing the data we propose the use of a new mixin that has an attribute to hold the data to pass to the image. We have a working modification of the OCCI-OS implementation that uses such mixin at https://github.com/enolfc/occi-os/tree/occi_user_data. In this case we have defined a mixin:
Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"
and
org.openstack.compute.user_data
as attribute (base64 encoded), that uses the user_data field of the OpenStack API.
We think a similar mixin could be added to OCCI to allow interoperability between resource providers.
The second part, related to retrieving the data, is more dependent of the back-end implementation. We have identified 3 different methods in the systems in place in FedCloud: - a metadata server (a la EC2), that located in a known URI servers the user-data - attach of an iso image to the VM that can be mounted - injection of a file with the user-data
All of those methods should be allowed and the VM once created should try them in order (the least invasive seems the metadata server, in our tests the file injection may fail with some VM images)
We think that the exact type and format of the data is not relevant. There could be specialized mixins to indicate explicitly that the data conforms to a executable script or to Cloud-init, but free-format data should also be allowed (for example at IFCA we have developed a system to contextualize just passing a JSON dictionary)
CIMI
We have deployed an instance of OpenStack for testing the CIMI implementation from this blueprint. A simple modification adds the support for the `userData` parameter in machine creation.
Sample code for the request (python code with requests package):
headers = {'content-type': 'application/json', 'accept': 'application/json', 'X-Auth-Token': keystone_token} img_body = { 'name': 'cimi-test-server', 'description': 'no desc', 'machineTemplate': { 'machineImage': {'href': image_id}, 'machineConfig': {'href': conf_id}, 'userData': base64.b64encode('this is just a test'), } } r = requests.post(cimi_url + '/machineCollection', headers=headers, data=json.dumps(img_body))
Implementation status
- OCCI-OS implementation is ready
- rOCCI not yet