Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Fedcloud-tf:WorkGroups:Contextualisation

From EGIWiki
Jump to navigation Jump to search
Main Roadmap and Innovation Technology For Users For Resource Providers Media


Workbenches: Open issues
Scenario 1
VM Management
Scenario 2
Data Management
Scenario 3
Information Systems
Scenario 4
Accounting
Scenario 5
Monitoring
Scenario 6
Notification
Scenario 7
Federated AAI
Scenario 8
VM Image Management
Scenario 9
Brokering
Scenario 10
Contextualisation
Scenario 11
Security



Leader: Enol Fernandez

Scope

This workgroup deals with the VM contextualization open issues, namely being able to pass user-defined data to the VM.

Roadmap

  • Evaluate mechanisms available in the cloud middleware to provide user-defined data
  • Evaluate possible OCCI extensions to pass user-defined data
  • Evaluate CIMI as alternative API (already includes user-data features)
  • Track implementation of OCCI extensions/CIMI on the cloud middleware

Cloud middleware support

OpenStack

OpenStack provides a meta-data server that provides instance-specific data. Instances can access the metadata service at http://169.254.169.254 using two different interfaces: one EC2-compatible and a proprietary Openstack interface.

The meta-data server provides the user public ssh-keys if included in the VM request and a user_data field that can be used freely by the user. cloud-init uses this field to contextualize machines.

OpenStack also allows file injection into the VM instance. These files can also be used for passing user-defined information to the running instance.

OpenNebula

OpenNebula provides contextualization information through an ISO image that is available in the VM instance. The contents of the ISO image include a context.sh file where a set of variables are defined. The contents of the context.sh are defined in the VM template.

A EC2-compatible meta-data server is also available at https://bitbucket.org/ricardoduarte/opennebula-metadata. It needs to be tested.

Stratuslab

StratusLab supports CloudInit, HEPiX and OpenNebula contextualization.

For HEPiX and OpenNebula contextualization, an ISO CDROM is attached to the VM that supports the file layout for both systems.

CloudInit contextualization handles web-server and disk based contextualization. StratusLab supports the second one. CloudInit only supports disks formatted as VFAT file systems at the moment. More information can be found at CloudInit in StratusLab documentation

Currently, the OpenNebula contextualization is the default. In the upcoming releases, the default will become CloudInit. All three methods for contextualization will be supported for the foreseeable future.

Okeanos/Synefo

Okeanos uses OpenStack API, in particular it supports the file-injection via http://docs.openstack.org/api/openstack-compute/2/content/Server_Personality-d1e2543.html

OCCI extension

OCCI does not include support for passing user-provided data to the instances.

We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined:

  • how to pass data upon VM creation
  • how to retrieve those data from the running VM

For passing the data we propose the use of a new mixin that has an attribute to hold the data to pass to the image. We have a working modification of the OCCI-OS implementation that uses such mixin at https://github.com/enolfc/occi-os/tree/occi_user_data. In this case we have defined a mixin:

Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"

and

org.openstack.compute.user_data

as attribute (base64 encoded), that uses the user_data field of the OpenStack API.

We think a similar mixin could be added to OCCI to allow interoperability between resource providers.

The second part, related to retrieving the data, is more dependent of the back-end implementation. We have identified 3 different methods in the systems in place in FedCloud: - a metadata server (a la EC2), that located in a known URI servers the user-data - attach of an iso image to the VM that can be mounted - injection of a file with the user-data

All of those methods should be allowed and the VM once created should try them in order (the least invasive seems the metadata server, in our tests the file injection may fail with some VM images)

We think that the exact type and format of the data is not relevant. There could be specialized mixins to indicate explicitly that the data conforms to a executable script or to Cloud-init, but free-format data should also be allowed (for example at IFCA we have developed a system to contextualize just passing a JSON dictionary)

CIMI

We have deployed an instance of OpenStack for testing the CIMI implementation from this blueprint. A simple modification adds the support for the `userData` parameter in machine creation.

Sample code for the request (python code with requests package):

headers = {'content-type': 'application/json',
                  'accept': 'application/json',
                  'X-Auth-Token': keystone_token}


 img_body = {
      'name': 'cimi-test-server',
      'description': 'no desc',
      'machineTemplate': {
          'machineImage': {'href': image_id},
          'machineConfig': {'href': conf_id},
          'userData': base64.b64encode('this is just a test'),
     }
 }
 
r = requests.post(cimi_url + '/machineCollection',
                            headers=headers,
                            data=json.dumps(img_body))

Implementation status

  • OCCI-OS implementation is ready
  • rOCCI not yet