Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Fedcloud-tf:WorkGroups:Contextualisation"

From EGIWiki
Jump to navigation Jump to search
 
(24 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}  
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}  


<font color="red">Leader: Enol Fernandez </font>
<font color="red"></font>
 
= [Closed] =
 
<font color="red">Leader: Enol Fernandez </font>  


== Scope  ==
== Scope  ==


This workgroup deals with the VM contextualization open issues, namely being able to pass user-defined data to the VM.
This workgroup deals with the VM contextualization open issues, namely being able to pass user-defined data to the VM.  


== Roadmap  ==
== Roadmap  ==


* Evaluate mechanisms available in the cloud middleware to provide user-defined data
*Evaluate mechanisms available in the cloud middleware to provide user-defined data  
* Evaluate possible OCCI extensions to pass user-defined data
*Evaluate possible OCCI extensions to pass user-defined data  
* Evaluate CIMI as alternative API (already includes user-data features)
*Evaluate CIMI as alternative API (already includes user-data features)  
* Track implementation of OCCI extensions/CIMI on the cloud middleware
*Track implementation of OCCI extensions/CIMI on the cloud middleware
 
== Contextualization  ==
 
Contextualization is the process of installing, configuring and preparing software upon boot time on a pre-defined virtual machine image (e.g. setting the hostname, IP addresses, SSH aithorized keys, starting services, installing applications, etc.). We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined:
 
*how to pass data upon VM creation (the exact type and format of the data should not be relevant, it should be up to the user)
*how to retrieve those data from the running VM
 
For passing the data we have proposed the use of a new OCCI mixin that has an attribute to hold the data to pass to the image. The second part, related to retrieving the data, is more dependent of the back-end implementation. There are different methods in the systems in place in FedCloud, but tools like cloud-init handle those possible differences in a transparent way for the users.
 
== OCCI support  ==
 
FedCloud has agreed on two mixins for pushing user defined data into the VM instances:
 
*<code><nowiki>Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"</nowiki></code>, and
*<code><nowiki>Category: public_key; scheme="http://schemas.openstack.org/instance/credentials#"; class="mixin"</nowiki></code>
 
The <code>user_data</code> mixin has an attribute <code>org.openstack.compute.user_data</code> where the use can specify the base64 encoded data that wants to be available at the VM upon instantiation.
 
In the case of the <code>public_key</code> mixin, there are two attributes to be specified:
 
*<code>org.openstack.credentials.publickey.name</code> with a name for the public key, and
*<code>org.openstack.credentials.publickey.data</code> with the public key itself.
 
These are specified as headers in the HTTP request for creating a VM. A sample curl request could look like this (credential managing is missing):
<pre>curl -H 'Category: compute;scheme="http://schemas.ogf.org/occi/infrastructure#";class="kind"' \
    -H 'Category: myVM;scheme="http://my.occi.service/occi/infrastructure/os_tpl#";class="mixin"' \
    -H 'Category: public_key;scheme="http://schemas.openstack.org/instance/credentials#";class="mixin"' \
    -H 'Category: user_data;scheme="http://schemas.openstack.org/compute/instance#";class="mixin"' \
    -H 'X-OCCI-Attribute: org.openstack.credentials.publickey.name="my_key"' \
    -H 'X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa BAA...zxe ==user@host" \
    -H 'X-OCCI-Attribute: org.openstack.compute.user_data="VGhpcyBpcyBhIHNpbXBsZSB0ZXN0IQ=="' \
    https://your.endpoint.com:1234/
</pre>
=== rOCCI client  ===
 
[https://github.com/gwdg/rOCCI-cli rOCCI client] includes support for both extensions via the <code>-T</code>/<code>--context</code> command line argument. The client does not need to previously b64encode the data. The curl command above could be translated into the following
<pre>occi -e https://your.endpoint.com:1234 --action create --resource compute \
    --attribute occi.core.title='ContextVM' --mixin os_tpl#myVM
    --context user_data='This is just a test!' --context public_key='ssh-rsa BAA...zxe ==user@host'
</pre>
the user data and public keys can be specified as files:
<pre>--context user_data="file:///path/to/file" --context public_key="file:///path/to/publickey"
</pre>
=== OCCI-OS (OpenStack)  ===
 
[https://github.com/EGI-FCTF/occi-os OCCI-OS] supports the extensions for pushing user data and keys for the <code>stable/folsom</code> and <code>stable/havana</code> versions.
 
=== rOCCI-server (OpenNebula and others)  ===
 
[http://gwdg.github.io/rOCCI-server/ rOCCI-server] also supports the extensions since version <code>4.0.0</code> of rOCCI framework.
 
=== Synnefo  ===
 
[https://www.synnefo.org/ Synnefo] also supports the extension in the latests versions.
 
== Using the data at the instances  ==
 
The proposed mixin does not impose any format on the data, thus the VM image creators are free to use this blob as it suites their needs. For most use cases, [https://help.ubuntu.com/community/CloudInit cloud-init] is the best way to avoid dealing with the details that each middleware has for providing the data in the VM instance.
 
=== cloud-init  ===
 
This task force recommends the use of cloud-init for the contextualization of VMs. Cloud-init frees the user from managing the specific ways for handling the contextualization information and it's widely available in most OS versions and IaaS cloud platforms. The latest versions support OpenNebula contextualization mechanisms. OpenStack and Synnefo contextualization are supported in most cloud-init versions (datasources are EC2 and NoCloud).
 
When creating a VM that is meant to run in different cloud middlewares leave the &lt;cloud&gt;datasource_list&lt;/cloud&gt; undefined (so it will try all the available ones), or set at least the following:
<pre>[ NoCloud, OpenNebula, Ec2, None ]
</pre>
<br> By default cloud-init will:
 
- put the ssh-key into the <code>~/.ssh/authorized_keys</code> of root user (or equivalent)
 
- if the user provided data is a script, it will be executed upon instantiation. More complex use-cases are supported, check [http://cloudinit.readthedocs.org/ cloud-init documentation] for examples.
 
<br>
 
==== cloud-init and OpenNebula  ====
 
The rOCCI server pushes into the OpenNebula context the base64 encoded user data provided by the client, however the default cloud-init is expecting clear text. This is fixed for the upcoming 0.7.5 version of cloud-init, in the meantime while it's released you can use the fedcloud packages (see below):
 
==== Packages  ====
 
Custom FedCloud packages for some distributions are available at [https://appdb.egi.eu/store/software/fedcloud.cloud.init EGI's AppDB FedCloud cloud-init]. These packages already manage the user data correctly for OpenNebula.
 
=== Contextualizing without cloudinit  ===
 
==== OpenStack  ====
 
OpenStack provides the user provided data at the meta-data server available at <code><nowiki>http://169.254.169.254</nowiki></code> for every instance. With the EC2 API, user data is available at <code><nowiki>http://169.254.169.254/latest/user-data</nowiki></code>, it can be fetched with curl:
<pre>curl http://169.254.169.254/latest/user-data
</pre>
OpenStack API provides the same data at <code><nowiki>http://169.254.169.254/openstack/latest/user_data</nowiki></code>.
 
Public keys are available at <code><nowiki>http://169.254.169.254/latest/meta-data/public-keys</nowiki></code> for EC2 API or <code><nowiki>http://169.254.169.254/openstack/latest/meta_data.json</nowiki></code>.
 
For getting OpenSSH key with EC2:
<pre>curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key
ssh-rsa  [...]==  xxx@host
&lt;pre&gt;
 
OpenStack API returns a json dictionary with a "public_keys" dictionary inside, where the available keys are stored:
&lt;pre&gt;
curl http://169.254.169.254/openstack/latest/meta_data.json
{... "public_keys": {"enolkey": "ssh-rsa [...]== xxx@host\n"}...}
</pre>
See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html and http://docs.openstack.org/user-guide/content/user-data.html for more info.
 
==== OpenNebula  ====
 
OpenNebula provides contextualization information through an ISO image that is available in the VM instance. The contents of the ISO image include a context.sh file where a set of variables are defined according to the VM template.
 
The iso filesystem is normally available at <code>/dev/sr0</code> where a <code>context.sh</code> file contains all the context variables declared in the OpenNebula template.
 
The <code>USER_DATA</code> variable contains a base64 enconded blob (can be decoded with <code>base64 -d</code>) with the data provided by the user with the OCCI client. The <code>SSH_PUBLIC_KEYS</code> variable contain the user public keys.
 
For using this context just mount the filesystem and source the context file:
<pre>$ mount -o ro /dev/sr0 /mnt/
$ cat /mnt/context.sh 
# Context variables generated by OpenNebula
DISK_ID='1'
ETH0_BROADCAST='193.144.35.127'
ETH0_DNS='193.144.34.209'
ETH0_GATEWAY='193.144.35.1'
ETH0_IP='193.144.35.111'
ETH0_MASK='255.255.255.128'
ETH0_NETWORK='193.144.35.128'
SSH_PUBLIC_KEY='ssh-rsa [...]== xxx@cloud'
TARGET='hdd'
TYPE='CDROM'
USER_DATA='VGhpcyBpcyBteSB1c2VyIGRhdGEK'
$ . /mnt/context.sh
$ echo $SSH_PUBLIC_KEY
ssh-rsa [...]== xxx@host
$ echo $USER_DATA | base64 -d
This is my user data
</pre>
See http://opennebula.org/documentation:archives:rel3.8:cong for more information.
 
==== Synnefo  ====
 
Synnefo provides context data in a [http://cloudinit.readthedocs.org/en/latest/topics/datasources.html#no-cloud cloud-init NoCloud] compatible format.
 
Two files are injected into the VMs: <code>/var/lib/cloud/seed/nocloud-net/meta-data</code>, which is a <code>yaml</code> file that contains the user provided public keys:
<pre>public-keys: ssh-rsa [...]== xxx@host
</pre>
And <code>/var/lib/cloud/seed/nocloud-net/user-data</code>, which contains a blob with the data provided by the user with the OCCI client.
 
== Setting up the middleware for contextualization  ==
 
=== OpenStack  ===
 
Default OpenStack installations do not require any additional steps for providing contextualization.
 
=== OpenNebula  ===
 
Cloud-init documentation includes [http://cloudinit.readthedocs.org/en/latest/topics/datasources.html#opennebula information on the OpenNebula configuration]. Check also OpenNebula [http://opennebula.org/documentation:documentation:context_overview contextualization overview] for more information.
 
== Windows Contextualization  ==
 
Windows VM also support contextualization of the machine on machine boot using [https://github.com/stackforge/cloudbase-init cloudbase-init] software. This software similarly to the cloud-init available for linux machines uses the available meta-data and user-data and executes a set of plugins to configure the virtual machine.
 
Cloudbase-init supports OpenStack meta-data and there is work in progress to support other cloud management frameworks, including OpenNebula.
 
=== Installation  ===
 
Cloudbase-init is distributed with a MSI installer build after every commit in the repo:
 
*[https://www.cloudbase.it/downloads/CloudbaseInitSetup_Beta_x64.msi 64-bit version]
*[https://www.cloudbase.it/downloads/CloudbaseInitSetup_Beta_x86.msi 32-bit version]
 
During the installation, you can configure:


== Cloud middleware support ==
*Name of a user that will be created and upon boot up (if it is an already existing user, cloudbase-init will only change its password)
*Network adapter / serial ports to use (defaults should be safe)


=== OpenStack ===
Normally is not needed to execute sysprep on the image after the installation of cloudbase-init
OpenStack provides a meta-data server that provides instance-specific data. Instances can access the metadata service at http://169.254.169.254 using two different interfaces: one EC2-compatible and a proprietary Openstack interface.


The meta-data server provides the user public ssh-keys if included in the VM request and a user_data field that can be used freely by the user. cloud-init uses this field to contextualize machines.
Once installed the machine can be uploaded/snapshoted to be started by the users.  


OpenStack also allows file injection into the VM instance. These files can also be used for passing user-defined information to the running instance.
=== Contextualization  ===


=== OpenNebula ===
The features of cloudbase-init are not as extensive as the Linux cloud-init, but it can still perform the actions described below. For an up-to-date list and documentation, check the [https://github.com/stackforge/cloudbase-init#plugins Plugin documentation of cloudbase-init], there is also documentation available at [http://wiki.cloudbase.it/cloudbaseinit-doc cloudbaseinit wiki]


OpenNebula provides contextualization information through an ISO image that is available in the VM instance. The contents of the ISO image include a context.sh file where a set of variables are defined. The contents of the context.sh are defined in the VM template.
==== User configuration  ====


A EC2-compatible meta-data server is also available at https://bitbucket.org/ricardoduarte/opennebula-metadata. It needs to be tested.
cloudbase-init by default will configure the user account configured during contextualization. Take into account that the configured group '''must''' exist before rebooting the machine.  


=== Stratuslab ===
===== Specifying the user password via OCCI  =====


StratusLab supports CloudInit, HEPiX and OpenNebula contextualization.
Cloudbase-init can set the user password if specified in the context of the VM. OpenStack does automatically generate a random password on machine creation but this is not available to the OCCI interface. We have created a new mixin that allows to specify the password you want the user to have as follows:


For HEPiX and OpenNebula contextualization, an ISO CDROM is attached to the VM that supports the file layout for both systems.
<code><nowiki>Category: admin_pwd; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"</nowiki></code>


CloudInit contextualization handles web-server and disk based contextualization. StratusLab supports the second one. CloudInit only supports disks formatted as VFAT file systems at the moment. More information can be found at CloudInit in StratusLab documentation
And the password is specified with the attribute:


Currently, the OpenNebula contextualization is the default. In the upcoming releases, the default will become CloudInit. All three methods for contextualization will be supported for the foreseeable future.
<code><nowiki>org.openstack.credentials.admin_pwd</nowiki></code>


=== Okeanos/Synefo ===
For using this mixin on VM creation, you can use a command line similar to this:
<pre>occi -e $OCCI_ENDPOINT -x $X509_USER_PROXY -X -n x509 \
    -a  create -r compute \
    -M $OS_TPL -M $RES_TPL \
    -t occi.core.title="$VM_TITLE" \
    -T user_data="$USER_DATA" \
    -M http://schemas.openstack.org/compute/instance#admin_pwd \
    -t org.openstack.credentials.admin_pwd="&lt;secret&gt;"
</pre>
==== User-data  ====


Okeanos uses OpenStack API, in particular it supports the file-injection via http://docs.openstack.org/api/openstack-compute/2/content/Server_Personality-d1e2543.html
cloudbase-init will try to execute as a script whatever is passed in the user-data. It uses the first line of the file to determine which kind of script is passed for execution:  


== OCCI extension ==
*For Windows batch, the first line of the file must be


OCCI does not include support for passing user-provided data to the instances.
rem cmd


We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined:
*For Powershell, the first line must be:
* how to pass data upon VM creation
* how to retrieve those data from the running VM


For passing the data we propose the use of a new mixin that has an attribute to hold the data to pass to the image. We have a working modification of the OCCI-OS implementation that uses such mixin at https://github.com/enolfc/occi-os/tree/occi_user_data. In this case we have defined a mixin:
  #ps1_sysnative
<pre>Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"</pre>
and <pre>org.openstack.compute.user_data</pre> as attribute (base64 encoded), that uses the user_data field of the OpenStack API.


We think a similar mixin could be added to OCCI to allow interoperability between resource providers.
or for a x86 PowerShell


The second part, related to retrieving the data, is more dependent of the back-end implementation. We have identified 3 different methods in the systems in place in FedCloud:
#ps1_x86
- a metadata server (a la EC2), that located in a known URI servers the user-data
- attach of an iso image to the VM that can be mounted
- injection of a file with the user-data


All of those methods should be allowed and the VM once created should try them in order (the least invasive seems the metadata server, in our tests the file injection may fail with some VM images)
*For Bash (it must be preinstalled on the VM!), the first line must ''begin'' with


We think that the exact type and format of the data is not relevant. There could be specialized mixins to indicate explicitly that the data conforms to a executable script or to Cloud-init, but free-format data should also be allowed (for example at IFCA we have developed a [https://github.com/enolfc/feynapps system to contextualize] just passing a JSON dictionary)
#!


== CIMI ==
for example, it can be


#!/bin/bash


We have deployed an instance of OpenStack for testing the CIMI implementation from this [https://blueprints.launchpad.net/nova/+spec/cimi blueprint].
The script will be executed with admin privileges. Take into account that the file must have DOS format, so if you are editing them on a UNIX-based machine you should make sure that the endline terminators are in the correct format.  
A simple [https://github.com/enolfc/cimi/commit/17635bf4fac24dd86af0fe86dd7300a5c588fded modification] adds the support for the `userData` parameter in machine creation.


Sample code for the request (python code with requests package):
==== Forcing re-execution of contextualization  ====


<nowiki>
cloudbase-init will execute the plugins only on the first time the image is booted. If you need to re-execute them you should remove in the Windows registry the following key:  
headers = {'content-type': 'application/json',
                  'accept': 'application/json',
                  'X-Auth-Token': keystone_token}


HKEY_LOCAL_MACHINE\Software\Cloudbase Solutions\Cloudbase-Init


img_body = {
on 64 bit versions of Windows, the key is:  
      'name': 'cimi-test-server',
      'description': 'no desc',
      'machineTemplate': {
          'machineImage': {'href': image_id},
          'machineConfig': {'href': conf_id},
          'userData': base64.b64encode('this is just a test'),
    }
}
r = requests.post(cimi_url + '/machineCollection',
                            headers=headers,
                            data=json.dumps(img_body))
</nowiki>


== Implementation status ==
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Cloudbase Solutions\Cloudbase-Init


* Both OCCI-OS and rOCCI server include support for
The next time cloudbase-init is executed (due to a reboot or by restarting the service), it will execute the plugins again.

Latest revision as of 09:27, 20 April 2015

Main Roadmap and Innovation Technology For Users For Resource Providers Media


Workbenches: Open issues
Scenario 1
VM Management
Scenario 2
Data Management
Scenario 3
Information Systems
Scenario 4
Accounting
Scenario 5
Monitoring
Scenario 6
Notification
Scenario 7
Federated AAI
Scenario 8
VM Image Management
Scenario 9
Brokering
Scenario 10
Contextualisation
Scenario 11
Security



[Closed]

Leader: Enol Fernandez

Scope

This workgroup deals with the VM contextualization open issues, namely being able to pass user-defined data to the VM.

Roadmap

  • Evaluate mechanisms available in the cloud middleware to provide user-defined data
  • Evaluate possible OCCI extensions to pass user-defined data
  • Evaluate CIMI as alternative API (already includes user-data features)
  • Track implementation of OCCI extensions/CIMI on the cloud middleware

Contextualization

Contextualization is the process of installing, configuring and preparing software upon boot time on a pre-defined virtual machine image (e.g. setting the hostname, IP addresses, SSH aithorized keys, starting services, installing applications, etc.). We have identified as a requirement to contextualize images the possibility of passing user provided data to the VM when they are instantiated. Hence there are two things to be defined:

  • how to pass data upon VM creation (the exact type and format of the data should not be relevant, it should be up to the user)
  • how to retrieve those data from the running VM

For passing the data we have proposed the use of a new OCCI mixin that has an attribute to hold the data to pass to the image. The second part, related to retrieving the data, is more dependent of the back-end implementation. There are different methods in the systems in place in FedCloud, but tools like cloud-init handle those possible differences in a transparent way for the users.

OCCI support

FedCloud has agreed on two mixins for pushing user defined data into the VM instances:

  • Category: user_data; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin", and
  • Category: public_key; scheme="http://schemas.openstack.org/instance/credentials#"; class="mixin"

The user_data mixin has an attribute org.openstack.compute.user_data where the use can specify the base64 encoded data that wants to be available at the VM upon instantiation.

In the case of the public_key mixin, there are two attributes to be specified:

  • org.openstack.credentials.publickey.name with a name for the public key, and
  • org.openstack.credentials.publickey.data with the public key itself.

These are specified as headers in the HTTP request for creating a VM. A sample curl request could look like this (credential managing is missing):

curl -H 'Category: compute;scheme="http://schemas.ogf.org/occi/infrastructure#";class="kind"' \
     -H 'Category: myVM;scheme="http://my.occi.service/occi/infrastructure/os_tpl#";class="mixin"' \
     -H 'Category: public_key;scheme="http://schemas.openstack.org/instance/credentials#";class="mixin"' \
     -H 'Category: user_data;scheme="http://schemas.openstack.org/compute/instance#";class="mixin"' \
     -H 'X-OCCI-Attribute: org.openstack.credentials.publickey.name="my_key"' \ 
     -H 'X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa BAA...zxe ==user@host" \
     -H 'X-OCCI-Attribute: org.openstack.compute.user_data="VGhpcyBpcyBhIHNpbXBsZSB0ZXN0IQ=="' \
     https://your.endpoint.com:1234/

rOCCI client

rOCCI client includes support for both extensions via the -T/--context command line argument. The client does not need to previously b64encode the data. The curl command above could be translated into the following

occi -e https://your.endpoint.com:1234 --action create --resource compute \
     --attribute occi.core.title='ContextVM' --mixin os_tpl#myVM 
     --context user_data='This is just a test!' --context public_key='ssh-rsa BAA...zxe ==user@host'

the user data and public keys can be specified as files:

--context user_data="file:///path/to/file" --context public_key="file:///path/to/publickey"

OCCI-OS (OpenStack)

OCCI-OS supports the extensions for pushing user data and keys for the stable/folsom and stable/havana versions.

rOCCI-server (OpenNebula and others)

rOCCI-server also supports the extensions since version 4.0.0 of rOCCI framework.

Synnefo

Synnefo also supports the extension in the latests versions.

Using the data at the instances

The proposed mixin does not impose any format on the data, thus the VM image creators are free to use this blob as it suites their needs. For most use cases, cloud-init is the best way to avoid dealing with the details that each middleware has for providing the data in the VM instance.

cloud-init

This task force recommends the use of cloud-init for the contextualization of VMs. Cloud-init frees the user from managing the specific ways for handling the contextualization information and it's widely available in most OS versions and IaaS cloud platforms. The latest versions support OpenNebula contextualization mechanisms. OpenStack and Synnefo contextualization are supported in most cloud-init versions (datasources are EC2 and NoCloud).

When creating a VM that is meant to run in different cloud middlewares leave the <cloud>datasource_list</cloud> undefined (so it will try all the available ones), or set at least the following:

[ NoCloud, OpenNebula, Ec2, None ]


By default cloud-init will:

- put the ssh-key into the ~/.ssh/authorized_keys of root user (or equivalent)

- if the user provided data is a script, it will be executed upon instantiation. More complex use-cases are supported, check cloud-init documentation for examples.


cloud-init and OpenNebula

The rOCCI server pushes into the OpenNebula context the base64 encoded user data provided by the client, however the default cloud-init is expecting clear text. This is fixed for the upcoming 0.7.5 version of cloud-init, in the meantime while it's released you can use the fedcloud packages (see below):

Packages

Custom FedCloud packages for some distributions are available at EGI's AppDB FedCloud cloud-init. These packages already manage the user data correctly for OpenNebula.

Contextualizing without cloudinit

OpenStack

OpenStack provides the user provided data at the meta-data server available at http://169.254.169.254 for every instance. With the EC2 API, user data is available at http://169.254.169.254/latest/user-data, it can be fetched with curl:

curl http://169.254.169.254/latest/user-data

OpenStack API provides the same data at http://169.254.169.254/openstack/latest/user_data.

Public keys are available at http://169.254.169.254/latest/meta-data/public-keys for EC2 API or http://169.254.169.254/openstack/latest/meta_data.json.

For getting OpenSSH key with EC2:

curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key
ssh-rsa  [...]==  xxx@host
<pre>

OpenStack API returns a json dictionary with a "public_keys" dictionary inside, where the available keys are stored:
<pre>
curl http://169.254.169.254/openstack/latest/meta_data.json
{... "public_keys": {"enolkey": "ssh-rsa [...]== xxx@host\n"}...}

See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html and http://docs.openstack.org/user-guide/content/user-data.html for more info.

OpenNebula

OpenNebula provides contextualization information through an ISO image that is available in the VM instance. The contents of the ISO image include a context.sh file where a set of variables are defined according to the VM template.

The iso filesystem is normally available at /dev/sr0 where a context.sh file contains all the context variables declared in the OpenNebula template.

The USER_DATA variable contains a base64 enconded blob (can be decoded with base64 -d) with the data provided by the user with the OCCI client. The SSH_PUBLIC_KEYS variable contain the user public keys.

For using this context just mount the filesystem and source the context file:

$ mount -o ro /dev/sr0 /mnt/
$ cat /mnt/context.sh  
# Context variables generated by OpenNebula
DISK_ID='1'
ETH0_BROADCAST='193.144.35.127'
ETH0_DNS='193.144.34.209'
ETH0_GATEWAY='193.144.35.1'
ETH0_IP='193.144.35.111'
ETH0_MASK='255.255.255.128'
ETH0_NETWORK='193.144.35.128'
SSH_PUBLIC_KEY='ssh-rsa [...]== xxx@cloud'
TARGET='hdd'
TYPE='CDROM'
USER_DATA='VGhpcyBpcyBteSB1c2VyIGRhdGEK'
$ . /mnt/context.sh
$ echo $SSH_PUBLIC_KEY
ssh-rsa [...]== xxx@host
$ echo $USER_DATA | base64 -d
This is my user data

See http://opennebula.org/documentation:archives:rel3.8:cong for more information.

Synnefo

Synnefo provides context data in a cloud-init NoCloud compatible format.

Two files are injected into the VMs: /var/lib/cloud/seed/nocloud-net/meta-data, which is a yaml file that contains the user provided public keys:

public-keys: ssh-rsa [...]== xxx@host

And /var/lib/cloud/seed/nocloud-net/user-data, which contains a blob with the data provided by the user with the OCCI client.

Setting up the middleware for contextualization

OpenStack

Default OpenStack installations do not require any additional steps for providing contextualization.

OpenNebula

Cloud-init documentation includes information on the OpenNebula configuration. Check also OpenNebula contextualization overview for more information.

Windows Contextualization

Windows VM also support contextualization of the machine on machine boot using cloudbase-init software. This software similarly to the cloud-init available for linux machines uses the available meta-data and user-data and executes a set of plugins to configure the virtual machine.

Cloudbase-init supports OpenStack meta-data and there is work in progress to support other cloud management frameworks, including OpenNebula.

Installation

Cloudbase-init is distributed with a MSI installer build after every commit in the repo:

During the installation, you can configure:

  • Name of a user that will be created and upon boot up (if it is an already existing user, cloudbase-init will only change its password)
  • Network adapter / serial ports to use (defaults should be safe)

Normally is not needed to execute sysprep on the image after the installation of cloudbase-init

Once installed the machine can be uploaded/snapshoted to be started by the users.

Contextualization

The features of cloudbase-init are not as extensive as the Linux cloud-init, but it can still perform the actions described below. For an up-to-date list and documentation, check the Plugin documentation of cloudbase-init, there is also documentation available at cloudbaseinit wiki

User configuration

cloudbase-init by default will configure the user account configured during contextualization. Take into account that the configured group must exist before rebooting the machine.

Specifying the user password via OCCI

Cloudbase-init can set the user password if specified in the context of the VM. OpenStack does automatically generate a random password on machine creation but this is not available to the OCCI interface. We have created a new mixin that allows to specify the password you want the user to have as follows:

Category: admin_pwd; scheme="http://schemas.openstack.org/compute/instance#"; class="mixin"

And the password is specified with the attribute:

org.openstack.credentials.admin_pwd

For using this mixin on VM creation, you can use a command line similar to this:

occi -e $OCCI_ENDPOINT -x $X509_USER_PROXY -X -n x509 \
     -a  create -r compute \
     -M $OS_TPL -M $RES_TPL \
     -t occi.core.title="$VM_TITLE" \
     -T user_data="$USER_DATA" \
     -M http://schemas.openstack.org/compute/instance#admin_pwd \
     -t org.openstack.credentials.admin_pwd="<secret>"

User-data

cloudbase-init will try to execute as a script whatever is passed in the user-data. It uses the first line of the file to determine which kind of script is passed for execution:

  • For Windows batch, the first line of the file must be
rem cmd
  • For Powershell, the first line must be:
#ps1_sysnative

or for a x86 PowerShell

#ps1_x86
  • For Bash (it must be preinstalled on the VM!), the first line must begin with
#!

for example, it can be

#!/bin/bash

The script will be executed with admin privileges. Take into account that the file must have DOS format, so if you are editing them on a UNIX-based machine you should make sure that the endline terminators are in the correct format.

Forcing re-execution of contextualization

cloudbase-init will execute the plugins only on the first time the image is booted. If you need to re-execute them you should remove in the Windows registry the following key:

HKEY_LOCAL_MACHINE\Software\Cloudbase Solutions\Cloudbase-Init

on 64 bit versions of Windows, the key is:

HKEY_LOCAL_MACHINE\Software\Wow6432Node\Cloudbase Solutions\Cloudbase-Init

The next time cloudbase-init is executed (due to a reboot or by restarting the service), it will execute the plugins again.