Difference between revisions of "Fedcloud-tf:WorkGroups:VM Marketplace"
m |
|||
Line 1: | Line 1: | ||
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}} | {{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}} | ||
<font color="red">Leader: Kostas Koumantaros, EGI-InSPIRE SA2 </font> | <font color="red">Leader: Kostas Koumantaros, EGI-InSPIRE SA2 </font> | ||
== Collaborators == | == Collaborators == | ||
Line 12: | Line 12: | ||
|- | |- | ||
| Scenario leader | | Scenario leader | ||
| EGI-InSPIRE SA2 | | EGI-InSPIRE SA2 | ||
| Kostas Koumantaros | | Kostas Koumantaros | ||
|- | |- | ||
Line 19: | Line 19: | ||
| Michel Jouvin | | Michel Jouvin | ||
|- | |- | ||
| Collaborator | | Collaborator | ||
| TCD | | TCD | ||
| Stuart Kenny | | Stuart Kenny | ||
|} | |} | ||
Line 26: | Line 26: | ||
== Roadmap == | == Roadmap == | ||
* Investigate how to do double endorsement | *Investigate how to do double endorsement | ||
*Investigate x509 + VOMS authentication | |||
== Scope == | |||
This workbench deals with the issues around setting up a VM Marketplace to: | |||
* | *Provide a publicly searchable place for VMs that may provide the application that is needed | ||
*Provide a common place to add a token of endorsement to a pertinent VM | |||
== | == Marketplace Howto == | ||
== Register an image with the EGI.eu Marketplace == | |||
(''Modified version of instructions compiled by Boris Parak. The original version can be found [http://meta.cesnet.cz/wiki/FedCloudDocumentation:How_to_upload_images_to_the_EGI.eu_Marketplace here]'') | |||
== | === Install and configure stratuslab-cli-tools === | ||
This part is very straight-forward, we need ''stratuslab-cli-tools''. So | |||
cd ~ | cd ~ | ||
mkdir stratuslab | mkdir stratuslab | ||
Line 48: | Line 52: | ||
wget http://repo.stratuslab.eu:8081/content/repositories/centos-6.2-releases/eu/stratuslab/pkgs/stratuslab-cli-user-pkg/2.2/stratuslab-cli-user-pkg-2.2.tar.gz | wget http://repo.stratuslab.eu:8081/content/repositories/centos-6.2-releases/eu/stratuslab/pkgs/stratuslab-cli-user-pkg/2.2/stratuslab-cli-user-pkg-2.2.tar.gz | ||
tar xvf stratuslab-cli-user-pkg-2.2.tar.gz | tar xvf stratuslab-cli-user-pkg-2.2.tar.gz | ||
and then conclude the installation process by appending the following to ''~/.bashrc'' | |||
and then conclude the installation process by appending the following to ''~/.bashrc'' | |||
# STRATUSLAB-CLI-TOOLS | # STRATUSLAB-CLI-TOOLS | ||
export PATH=$PATH:~/stratuslab/bin | export PATH=$PATH:~/stratuslab/bin | ||
export PYTHONPATH=$PYTHONPATH:~/stratuslab/lib/stratuslab/python | export PYTHONPATH=$PYTHONPATH:~/stratuslab/lib/stratuslab/python | ||
RPMs for the client are also available from the StratusLab yum repositories, see http://yum.stratuslab.eu/. Packages are provided for CentOS 6.2, OpenSuse 12.1 and Fedora 16. | RPMs for the client are also available from the StratusLab yum repositories, see http://yum.stratuslab.eu/. Packages are provided for CentOS 6.2, OpenSuse 12.1 and Fedora 16. | ||
=== Get demo images === | === Get demo images === | ||
=== Upload the image into your cloud === | There are two images required for the demo. Each resource provider should upload a metadata entry for each. The first is the BNCweb image, which is available from https://appliance-repo.egi.eu/images/base/egi-bncweb/1.0/egi-bncweb.img. The second is a plain Debian 6 image (https://appliance-repo.egi.eu/images/base/Debian-6.0.5-x86_64-base/1.0/debian-6.0.5-x86_64-base.img). | ||
This step is different for every cloud platform. For instance, in OpenNebula v3.4+ you can use Sunstone GUI to upload images directly, in previous versions you have to upload the image to the frontend and then register it. | |||
=== Upload the image into your cloud === | |||
This step is different for every cloud platform. For instance, in OpenNebula v3.4+ you can use Sunstone GUI to upload images directly, in previous versions you have to upload the image to the frontend and then register it. | |||
Sice FedCloud-TF will be using OCCI to access the cloud, you must provide a location of the image that is OCCI-compatible. To find the right link you can browse through all the storage elements registered in your OCCI server | |||
https://occi.host:port/storage/ | https://occi.host:port/storage/ | ||
checking the ''occi.core.title'' attribute for the right name. You should end up with something like | |||
checking the ''occi.core.title'' attribute for the right name. You should end up with something like | |||
https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511 | https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511 | ||
=== Build the metadata === | === Build the metadata === | ||
The EGI.eu Marketplace stores only metadata which points to the image, provide basic information and integrity verification. Since RDF is not the most user-friendly format, we can use ''stratus-build-metadata'' to generate a template | |||
The EGI.eu Marketplace stores only metadata which points to the image, provide basic information and integrity verification. Since RDF is not the most user-friendly format, we can use ''stratus-build-metadata'' to generate a template | |||
stratus-build-metadata --author='##YOUR_NAME##' --type=base --os=Ubuntu --os-version=11.04 --os-arch=x86_64 \ | stratus-build-metadata --author='##YOUR_NAME##' --type=base --os=Ubuntu --os-version=11.04 --os-arch=x86_64 \ | ||
--image-version=1.0 --hypervisor=xen --format=raw --comment='BNCWeb appliance for the OGF35 demo available at ##YOUR_SITE##' \ | --image-version=1.0 --hypervisor=xen --format=raw --comment='BNCWeb appliance for the OGF35 demo available at ##YOUR_SITE##' \ | ||
--compression=none --location='https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511' egi-bncweb.img | --compression=none --location='https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511' egi-bncweb.img | ||
=== Modify the metadata === | '''Note:''' stratus-build-metadata needs the image to compute checksums, you can download it here [https://appliance-repo.egi.eu/images/base/egi-bncweb/1.0/egi-bncweb.img egi-bncweb.img] | ||
=== Modify the metadata === | |||
Now we can check/modify the metadata, the most important elements are ''dcterms:valid'' and ''dcterms:title''. | Now we can check/modify the metadata, the most important elements are ''dcterms:valid'' and ''dcterms:title''. | ||
The correct format for ''dcterms:title'' is ''EGI-##IMAGE_NAME##-##SITE_NAME##''. This field will need to be manually added to the metadata file. You can also modify the validity date as required | The correct format for ''dcterms:title'' is ''EGI-##IMAGE_NAME##-##SITE_NAME##''. This field will need to be manually added to the metadata file. You can also modify the validity date as required. | ||
<pre> | '''Metadata from the EGI.eu Marketplace cannot be removed, it can only expire.''' It is also possible to ''deprecate'' an entry. This might be necessary, if for example, a security issue is detected with the image, or if you simply wish to no longer endorse the image. Instructions for the stratus-deprecate-image command can be found [http://stratuslab.eu/doku.php/ref-doc:user-cli#stratus-deprecate-metadata here]. | ||
<pre><?xml version="1.0" encoding="UTF-8" standalone="no"?> | |||
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" | |||
xmlns:dcterms="http://purl.org/dc/terms/" | xmlns:dcterms="http://purl.org/dc/terms/" | ||
xmlns:slterms="http://mp.stratuslab.eu/slterms#" | xmlns:slterms="http://mp.stratuslab.eu/slterms#" | ||
xmlns:slreq="http://mp.stratuslab.eu/slreq#" | xmlns:slreq="http://mp.stratuslab.eu/slreq#" | ||
xml:base="http://mp.stratuslab.eu/" | xml:base="http://mp.stratuslab.eu/"> | ||
<rdf:Description rdf:about="#DtRwHZzoo1xFKtk-iL51t6RNQ9Q"> | |||
<dcterms:identifier>DtRwHZzoo1xFKtk-iL51t6RNQ9Q</dcterms:identifier> | |||
<slreq:bytes>14680064000</slreq:bytes> | |||
<slreq:checksum rdf:parseType="Resource"> | |||
<slreq:algorithm>MD5</slreq:algorithm> | |||
<slreq:value>144fff2477673aa1d883f0a3ba89f273</slreq:value> | |||
</slreq:checksum> | |||
<slreq:checksum rdf:parseType="Resource"> | |||
<slreq:algorithm>SHA-1</slreq:algorithm> | |||
<slreq:value>3b51c07673a28d7114ab64fa22f9d6de91350f50</slreq:value> | |||
</slreq:checksum> | |||
<slreq:checksum rdf:parseType="Resource"> | |||
<slreq:algorithm>SHA-256</slreq:algorithm> | |||
<slreq:value>8bde348c81e5a2aa5aa51b8d39a30ad137d0482decd5960cd95594d224a45bdd</slreq:value> | |||
</slreq:checksum> | |||
<slreq:checksum rdf:parseType="Resource"> | |||
<slreq:algorithm>SHA-512</slreq:algorithm> | |||
<slreq:value>e780f2aa6922bc7cfdaae4a5e410f6b499bef5c83314bcd760b082b625860834c4942de9d096c7aa83cdad0411c47686f2e7d0fcc65f816475f6525db28b236d</slreq:value> | |||
</slreq:checksum> | |||
<slreq:endorsement rdf:parseType="Resource"/> | |||
<dcterms:title>EGI-BNCweb-##YOUR_SITE##</dcterms:title> | |||
<dcterms:type>base</dcterms:type> | |||
<slterms:kind>machine</slterms:kind> | |||
<slterms:os>Ubuntu</slterms:os> | |||
<slterms:os-version>11.04</slterms:os-version> | |||
<slterms:os-arch>x86_64</slterms:os-arch> | |||
<slterms:version>1.0</slterms:version> | |||
<dcterms:compression>none</dcterms:compression> | |||
<slterms:location>https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511</slterms:location> | |||
<dcterms:format>raw</dcterms:format> | |||
<dcterms:creator>##YOUR_NAME##</dcterms:creator> | |||
<dcterms:created>2012-06-12T12:36:25Z</dcterms:created> | |||
<dcterms:valid>2012-06-14T12:36:25Z</dcterms:valid> | |||
<dcterms:description>BNCWeb appliance for the OGF35 demo available at ##YOUR_SITE##</dcterms:description> | |||
<slterms:hypervisor>xen</slterms:hypervisor> | |||
<dcterms:publisher>##YOUR_SITE##</dcterms:publisher> | |||
</rdf:Description> | |||
</rdf:RDF> | |||
</pre> | </pre> | ||
=== Sign the metadata === | |||
To establish the origin of the image, we have to sign the metadata with a personal certificate (ideally the one registered with EGI.eu). '''Before''' doing this you should familiarise yourself with the [https://documents.egi.eu/public/ShowDocument?docid=771 EGI Security Policy for the Endorsement and Operation of Virtual Machine Images]. | |||
To establish the origin of the image, we have to sign the metadata with a personal certificate (ideally the one registered with EGI.eu). '''Before''' doing this you should familiarise yourself with the [https://documents.egi.eu/public/ShowDocument?docid=771 EGI Security Policy for the Endorsement and Operation of Virtual Machine Images]. | |||
stratus-sign-metadata --p12-cert=##FULL_PATH_TO_usercred.p12## egi-bncweb.xml | stratus-sign-metadata --p12-cert=##FULL_PATH_TO_usercred.p12## egi-bncweb.xml | ||
=== Register the metadata with the EGI.eu Marketplace === | === Register the metadata with the EGI.eu Marketplace === | ||
And to complete the process, we have to upload the metadata to the EGI.eu Marketplace with ''stratus-upload-metadata'' | |||
And to complete the process, we have to upload the metadata to the EGI.eu Marketplace with ''stratus-upload-metadata'' | |||
stratus-upload-metadata --marketplace-endpoint=marketplace.egi.eu egi-bncweb.xml | stratus-upload-metadata --marketplace-endpoint=marketplace.egi.eu egi-bncweb.xml | ||
or manually at | |||
or manually at | |||
http://marketplace.egi.eu/upload | http://marketplace.egi.eu/upload |
Revision as of 12:57, 23 August 2012
Main | Roadmap and Innovation | Technology | For Users | For Resource Providers | Media |
Leader: Kostas Koumantaros, EGI-InSPIRE SA2
Collaborators
Role | Institution | Name |
---|---|---|
Scenario leader | EGI-InSPIRE SA2 | Kostas Koumantaros |
Collaborator | GRIF | Michel Jouvin |
Collaborator | TCD | Stuart Kenny |
Roadmap
- Investigate how to do double endorsement
- Investigate x509 + VOMS authentication
Scope
This workbench deals with the issues around setting up a VM Marketplace to:
- Provide a publicly searchable place for VMs that may provide the application that is needed
- Provide a common place to add a token of endorsement to a pertinent VM
Marketplace Howto
Register an image with the EGI.eu Marketplace
(Modified version of instructions compiled by Boris Parak. The original version can be found here)
Install and configure stratuslab-cli-tools
This part is very straight-forward, we need stratuslab-cli-tools. So
cd ~ mkdir stratuslab cd stratuslab wget http://repo.stratuslab.eu:8081/content/repositories/centos-6.2-releases/eu/stratuslab/pkgs/stratuslab-cli-user-pkg/2.2/stratuslab-cli-user-pkg-2.2.tar.gz tar xvf stratuslab-cli-user-pkg-2.2.tar.gz
and then conclude the installation process by appending the following to ~/.bashrc
# STRATUSLAB-CLI-TOOLS export PATH=$PATH:~/stratuslab/bin export PYTHONPATH=$PYTHONPATH:~/stratuslab/lib/stratuslab/python
RPMs for the client are also available from the StratusLab yum repositories, see http://yum.stratuslab.eu/. Packages are provided for CentOS 6.2, OpenSuse 12.1 and Fedora 16.
Get demo images
There are two images required for the demo. Each resource provider should upload a metadata entry for each. The first is the BNCweb image, which is available from https://appliance-repo.egi.eu/images/base/egi-bncweb/1.0/egi-bncweb.img. The second is a plain Debian 6 image (https://appliance-repo.egi.eu/images/base/Debian-6.0.5-x86_64-base/1.0/debian-6.0.5-x86_64-base.img).
Upload the image into your cloud
This step is different for every cloud platform. For instance, in OpenNebula v3.4+ you can use Sunstone GUI to upload images directly, in previous versions you have to upload the image to the frontend and then register it.
Sice FedCloud-TF will be using OCCI to access the cloud, you must provide a location of the image that is OCCI-compatible. To find the right link you can browse through all the storage elements registered in your OCCI server
https://occi.host:port/storage/
checking the occi.core.title attribute for the right name. You should end up with something like
https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511
Build the metadata
The EGI.eu Marketplace stores only metadata which points to the image, provide basic information and integrity verification. Since RDF is not the most user-friendly format, we can use stratus-build-metadata to generate a template
stratus-build-metadata --author='##YOUR_NAME##' --type=base --os=Ubuntu --os-version=11.04 --os-arch=x86_64 \ --image-version=1.0 --hypervisor=xen --format=raw --comment='BNCWeb appliance for the OGF35 demo available at ##YOUR_SITE##' \ --compression=none --location='https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511' egi-bncweb.img
Note: stratus-build-metadata needs the image to compute checksums, you can download it here egi-bncweb.img
Modify the metadata
Now we can check/modify the metadata, the most important elements are dcterms:valid and dcterms:title.
The correct format for dcterms:title is EGI-##IMAGE_NAME##-##SITE_NAME##. This field will need to be manually added to the metadata file. You can also modify the validity date as required.
Metadata from the EGI.eu Marketplace cannot be removed, it can only expire. It is also possible to deprecate an entry. This might be necessary, if for example, a security issue is detected with the image, or if you simply wish to no longer endorse the image. Instructions for the stratus-deprecate-image command can be found here.
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:slterms="http://mp.stratuslab.eu/slterms#" xmlns:slreq="http://mp.stratuslab.eu/slreq#" xml:base="http://mp.stratuslab.eu/"> <rdf:Description rdf:about="#DtRwHZzoo1xFKtk-iL51t6RNQ9Q"> <dcterms:identifier>DtRwHZzoo1xFKtk-iL51t6RNQ9Q</dcterms:identifier> <slreq:bytes>14680064000</slreq:bytes> <slreq:checksum rdf:parseType="Resource"> <slreq:algorithm>MD5</slreq:algorithm> <slreq:value>144fff2477673aa1d883f0a3ba89f273</slreq:value> </slreq:checksum> <slreq:checksum rdf:parseType="Resource"> <slreq:algorithm>SHA-1</slreq:algorithm> <slreq:value>3b51c07673a28d7114ab64fa22f9d6de91350f50</slreq:value> </slreq:checksum> <slreq:checksum rdf:parseType="Resource"> <slreq:algorithm>SHA-256</slreq:algorithm> <slreq:value>8bde348c81e5a2aa5aa51b8d39a30ad137d0482decd5960cd95594d224a45bdd</slreq:value> </slreq:checksum> <slreq:checksum rdf:parseType="Resource"> <slreq:algorithm>SHA-512</slreq:algorithm> <slreq:value>e780f2aa6922bc7cfdaae4a5e410f6b499bef5c83314bcd760b082b625860834c4942de9d096c7aa83cdad0411c47686f2e7d0fcc65f816475f6525db28b236d</slreq:value> </slreq:checksum> <slreq:endorsement rdf:parseType="Resource"/> <dcterms:title>EGI-BNCweb-##YOUR_SITE##</dcterms:title> <dcterms:type>base</dcterms:type> <slterms:kind>machine</slterms:kind> <slterms:os>Ubuntu</slterms:os> <slterms:os-version>11.04</slterms:os-version> <slterms:os-arch>x86_64</slterms:os-arch> <slterms:version>1.0</slterms:version> <dcterms:compression>none</dcterms:compression> <slterms:location>https://occi.host:port/storage/a39a1d08-bff8-5a62-ba68-a1cd76bb4511</slterms:location> <dcterms:format>raw</dcterms:format> <dcterms:creator>##YOUR_NAME##</dcterms:creator> <dcterms:created>2012-06-12T12:36:25Z</dcterms:created> <dcterms:valid>2012-06-14T12:36:25Z</dcterms:valid> <dcterms:description>BNCWeb appliance for the OGF35 demo available at ##YOUR_SITE##</dcterms:description> <slterms:hypervisor>xen</slterms:hypervisor> <dcterms:publisher>##YOUR_SITE##</dcterms:publisher> </rdf:Description> </rdf:RDF>
Sign the metadata
To establish the origin of the image, we have to sign the metadata with a personal certificate (ideally the one registered with EGI.eu). Before doing this you should familiarise yourself with the EGI Security Policy for the Endorsement and Operation of Virtual Machine Images.
stratus-sign-metadata --p12-cert=##FULL_PATH_TO_usercred.p12## egi-bncweb.xml
Register the metadata with the EGI.eu Marketplace
And to complete the process, we have to upload the metadata to the EGI.eu Marketplace with stratus-upload-metadata
stratus-upload-metadata --marketplace-endpoint=marketplace.egi.eu egi-bncweb.xml
or manually at
http://marketplace.egi.eu/upload