Difference between revisions of "Fedcloud-tf:WorkGroups:Scenario1:OpenNebulaInstallation"
(Created page with "== OpenNebula == === rOCCI === This section describes how to intall and configure rOCCI 0.5.x in SL6 ==== Requirements ==== The following setup is recommended *usage of the R...") |
|||
Line 1: | Line 1: | ||
== OpenNebula == | == OpenNebula == | ||
=== rOCCI === | |||
This section describes how to intall and configure rOCCI 0.5.x in SL6 | |||
==== Requirements ==== | |||
The following setup is recommended | |||
The following setup is recommended | |||
*usage of the Ruby Version Manger (RVM) | *usage of the Ruby Version Manger (RVM) | ||
*Ruby | *Ruby >= 1.9.3 | ||
*Bundler gem installed (use ''gem install bundler'') | *Bundler gem installed (use ''gem install bundler'') | ||
==== Installation ==== | ==== Installation ==== | ||
===== Apache and EMI2 middleware ===== | |||
*Install EMI2 gridsite packages: | *Install EMI2 gridsite packages: | ||
Line 26: | Line 28: | ||
# chkconfig httpd on | # chkconfig httpd on | ||
<br> | |||
===== RVM ===== | ===== RVM ===== | ||
*As oneadmin run : | *As oneadmin run : | ||
$ curl -L https://get.rvm.io | sudo bash -s stable | $ curl -L https://get.rvm.io | sudo bash -s stable | ||
<br> | |||
<br> | |||
*And now add oneadmin user to RVM group: | |||
# usermod -a -G rvm oneadmin | # usermod -a -G rvm oneadmin | ||
<br> | |||
*Now as oneadmin user: | *Now as oneadmin user: | ||
$ source /etc/profile.d/rvm.sh | $ source /etc/profile.d/rvm.sh | ||
$ rvm user gemsets | $ rvm user gemsets | ||
<br> | |||
*Checkout latest version from GIT: | |||
$ git clone git://github.com/gwdg/rOCCI-server.git | $ git clone git://github.com/gwdg/rOCCI-server.git | ||
<br> | |||
*Change to rOCCI folder: | *Change to rOCCI folder: | ||
Line 55: | Line 63: | ||
$ cd rOCCI-server | $ cd rOCCI-server | ||
<br> | |||
*Use RVM: | *Use RVM: | ||
Line 60: | Line 69: | ||
$ rvm install ruby-1.9.3 | $ rvm install ruby-1.9.3 | ||
<br> In a different console install rvm deps if needed: | |||
In a different console install rvm deps if needed: | |||
*Install dependencies before press 'q', from another console, for SL machine: | *Install dependencies before press 'q', from another console, for SL machine: | ||
Line 67: | Line 75: | ||
#yum install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion libexpat1-dev rubygem-xmlparser expat expat-devel | #yum install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion libexpat1-dev rubygem-xmlparser expat expat-devel | ||
<br> | |||
*Install gems and rocci service (v0.5.x in this case): | *Install gems and rocci service (v0.5.x in this case): | ||
Line 73: | Line 82: | ||
$ bundle install | $ bundle install | ||
<br> | |||
<br> | |||
==== Configuration ==== | |||
rOCCI-server comes with different backends. Check the etc folder for available backends (e.g. dummy, opennebula, ...). Each backend has an example configuration in a file with the name of the backend and the extension .json. Copy one of those files to etc/backend/default.json and adapt its content to your setting. | |||
To configure the behaviour of compute, network and storage resource creation, edit the backend specific extensions of the OCCI model at etc/backend/$BACKEND/model (e.g. etc/backend/dummy/model for the dummy backend). | |||
To change the predefined resource or OS templates, you can adapt the existing templates in etc/backend/$BACKEND/templates or add new templates. If resource or OS templates are already registered in the backend, they will be automatically discovered by rOCCI-server. | |||
To change the predefined resource or OS templates, you can adapt the existing templates in etc/backend/$BACKEND/templates or add new templates. If resource or OS templates are already registered in the backend, they will be automatically discovered by rOCCI-server. | |||
<br> | |||
$ cd /var/lib/one/rOCCI-server/etc/backend/opennebula | $ cd /var/lib/one/rOCCI-server/etc/backend/opennebula | ||
<br> | |||
*And now copy opennebula backend before edit it: | *And now copy opennebula backend before edit it: | ||
Line 94: | Line 105: | ||
$ cd .. | $ cd .. | ||
<br> | |||
*And we change it to use our configuration | *And we change it to use our configuration | ||
{ | { | ||
"resources":[ | |||
{ | |||
"kind":"http://rocci.info/server/backend#opennebula", | |||
"attributes":{ | |||
"info":{ | |||
"rocci":{ | |||
"backend":{ | |||
"opennebula":{ | |||
"endpoint":"http://localhost:2633/RPC2", | |||
"admin":"rocci", | |||
"password":"*****", | |||
"scheme":"http://my.occi.service/" | |||
} | |||
} | |||
} | |||
} | |||
} | } | ||
} | |||
] | |||
} | |||
<br> | |||
*Edit ''/var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates/compute.erb'' file to fit your cloud resources: | *Edit ''/var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates/compute.erb'' file to fit your cloud resources: | ||
<br> | |||
===== OpenNebula backend ===== | ===== OpenNebula backend ===== | ||
If you want to change the actual deployment within OpenNebula you can change the OpenNebula templates in the files in /var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates. | If you want to change the actual deployment within OpenNebula you can change the OpenNebula templates in the files in /var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates. | ||
To configure OpenNebula resource templates (e.g. small, medium, large, ...) change the files in etc/backend/opennebula/templates . | |||
*As example to use x86_64 arch by default edit /var/lib/one/rOCCI-server/etc/backend/opennebula/templates and s change this line for these files (for small/medium/large): | *As example to use x86_64 arch by default edit /var/lib/one/rOCCI-server/etc/backend/opennebula/templates and s change this line for these files (for small/medium/large): | ||
... | ... | ||
"architecture":{ | |||
"Mutable":true, | |||
"Required":false, | |||
"Type":"string", | |||
"Pattern":"x86|x64", | |||
"Default":"x64" | |||
... | ... | ||
<br> | |||
===== Add rocci user ===== | ===== Add rocci user ===== | ||
The Sunstone, EC2 and OCCI services communicate with the core using a “server” user. OpenNebula creates the serveradmin account at bootstrap, with the authentication driver server_cipher (symmetric key). | |||
The Sunstone, EC2 and OCCI services communicate with the core using a “server” user. OpenNebula creates the serveradmin account at bootstrap, with the authentication driver server_cipher (symmetric key). | |||
This “server” user uses a special authentication mechanism that allows the servers to perform an operation on behalf of other user. | This “server” user uses a special authentication mechanism that allows the servers to perform an operation on behalf of other user. | ||
Line 148: | Line 166: | ||
*To add rocci user we run as oneadmin: | *To add rocci user we run as oneadmin: | ||
$ oneuser create rocci | $ oneuser create rocci <good_pass_here_from_default.json> --driver server_cipher | ||
$ oneuser chgrp rocci oneadmin | $ oneuser chgrp rocci oneadmin | ||
<br> (!) User/Pass is the same that default.json conf file. | |||
<br> | |||
===== Apache conf ===== | ===== Apache conf ===== | ||
/!\ add apache user to cloud or oneadmin group: | /!\ add apache user to cloud or oneadmin group: | ||
Line 161: | Line 179: | ||
# usermod -a -G cloud apache | # usermod -a -G cloud apache | ||
Let passenger guide you through installing and or configuring Apache for you | Let passenger guide you through installing and or configuring Apache for you | ||
*/!\ '''Add oneadmin to sudoers (/etc/sudoers) and for SL and RH flavors also set secure_path value, as example for ruby-1.9.3 for SL6:''' | */!\ '''Add oneadmin to sudoers (/etc/sudoers) and for SL and RH flavors also set secure_path value, as example for ruby-1.9.3 for SL6:''' | ||
Line 173: | Line 191: | ||
.. | .. | ||
<br> | |||
*If you want to configure apache automatically, from /var/lib/one/rOCCI-server directory as oneadmin user run this ( {i} for SL6 installation these packages are required: ''apache2-prefork-dev libapr1-dev libaprutil1-dev'') | *If you want to configure apache automatically, from /var/lib/one/rOCCI-server directory as oneadmin user run this ( {i} for SL6 installation these packages are required: ''apache2-prefork-dev libapr1-dev libaprutil1-dev'') | ||
Line 180: | Line 198: | ||
$ bundle exec rvmsudo passenger-install-apache2-module | $ bundle exec rvmsudo passenger-install-apache2-module | ||
<br> | |||
*This script shows this info at the end: | *This script shows this info at the end: | ||
<br> | |||
The Apache 2 module was successfully installed. | The Apache 2 module was successfully installed. | ||
Line 190: | Line 209: | ||
LoadModule passenger_module /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12/ext/apache2/mod_passenger.so | LoadModule passenger_module /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12/ext/apache2/mod_passenger.so | ||
PassengerRoot /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12 | |||
PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby | |||
After you restart Apache, you are ready to deploy any number of Ruby on Rails | After you restart Apache, you are ready to deploy any number of Ruby on Rails | ||
Line 197: | Line 216: | ||
configuration! | configuration! | ||
<br> | |||
(!) Now we must create '''/etc/httpd/conf.d/passenger.conf''' file | |||
(!) Now we must create '''/etc/httpd/conf.d/passenger.conf''' file | |||
*as root: ''vi /etc/httpd/conf.d/passenger.conf'' to include these lines | *as root: ''vi /etc/httpd/conf.d/passenger.conf'' to include these lines | ||
Line 208: | Line 227: | ||
PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby | PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby | ||
<br> | |||
---- | ---- | ||
<br> | |||
*Manually check these files (In this case we will use 3202 port for rOCCI service): | *Manually check these files (In this case we will use 3202 port for rOCCI service): | ||
edit /etc/httpd/conf.d/ssl.conf: | edit /etc/httpd/conf.d/ssl.conf: | ||
LoadModule ssl_module modules/mod_ssl.so | LoadModule ssl_module modules/mod_ssl.so | ||
Line 224: | Line 244: | ||
.. | .. | ||
<br> | |||
Create a new VirtualHost in the conf.d directory of Apache (e.g. in /etc/httpd/conf.d/rocci.conf) with the following content: | |||
<br> | |||
<VirtualHost your.host.name:3202> | |||
ServerName your.host.name | |||
<br> | |||
SSLEngine on | SSLEngine on | ||
SSLProtocol all | |||
SSLCertificateFile /etc/grid-security/hostcert.pem | |||
SSLCertificateKeyFile /etc/grid-security/hostkey.pem | |||
SSLProxyEngine on | SSLProxyEngine on | ||
SSLCACertificatePath /etc/grid-security/certificates | |||
SSLCARevocationPath /etc/grid-security/certificates | |||
<br> | |||
SSLVerifyClient require | SSLVerifyClient require | ||
SSLVerifyDepth 10 | |||
SSLOptions +StdEnvVars | |||
# important, this needs to point to the public folder of your rOCCI-server | # important, this needs to point to the public folder of your rOCCI-server | ||
DocumentRoot /var/lib/one/rOCCI-server/public | |||
<Directory /opt/rOCCI-server/public> | |||
## This exports various bits of info into the CGI environment | ## This exports various bits of info into the CGI environment | ||
## variables (and is needed for gridsite-admin.cgi to work.) | |||
GridSiteEnvs on | |||
## Nice GridSite directory listings (without truncating file names!) | ## Nice GridSite directory listings (without truncating file names!) | ||
GridSiteIndexes off | |||
## If this is greater than zero, we will accept GSI Proxies for clients | ## If this is greater than zero, we will accept GSI Proxies for clients | ||
## (full client certificates - eg inside web browsers - are always ok) | ## (full client certificates - eg inside web browsers - are always ok) | ||
GridSiteGSIProxyLimit 1 | |||
## This directive allows authorized people to write/delete files | ## This directive allows authorized people to write/delete files | ||
## from non-browser clients - eg with htcp(1) | |||
GridSiteMethods "" | |||
Allow from all | Allow from all | ||
Options -MultiViews | |||
</Directory> | |||
LogLevel debug | LogLevel debug | ||
</VirtualHost> | |||
<br> /!\ '''Change /var/lib/one/rOCCI-server/public permissions as oneadmin:''' | |||
/!\ '''Change /var/lib/one/rOCCI-server/public permissions as oneadmin:''' | |||
$ chmod -R 770 /var/lib/one/rOCCI-server/public | $ chmod -R 770 /var/lib/one/rOCCI-server/public | ||
$ chmod 440 /etc/grid-security/hostkey.pem | $ chmod 440 /etc/grid-security/hostkey.pem | ||
<br> /!\ Include apache user into cloud group: | |||
/!\ Include apache user into cloud group: | |||
# usermod -a -G cloud apache | # usermod -a -G cloud apache | ||
<br> | |||
<br> /!\ '''You have to start/restart Apache before you can use rOCCI-server!''' | |||
*Make sure that can validate fedcloud.egi.eu's certs, i.e. the following file exists: | |||
/ | # cat /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.egee.cesnet.cz.lsc | ||
/DC=org/DC=terena/DC=tcs/C=CZ/O=CESNET/CN=voms1.egee.cesnet.cz | |||
/C=NL/O=TERENA/CN=TERENA eScience SSL CA | |||
<br> | |||
*Finally: | |||
/etc/init.d/httpd restart | /etc/init.d/httpd restart | ||
<br> | |||
==== rOCCI + VOMS ==== | ==== rOCCI + VOMS ==== | ||
If you want to use X.509/VOMS authentication for your users, you need to create the users in OpenNebula with the X.509 driver. For a user named 'johnsmith' from fedcloud.egi.eu the command may look like this | |||
<br> | |||
$ oneuser create johnsmith "/DC=es/DC=irisgrid/O=cesga/CN=johnsmith|/DC=es/DC=irisgrid/O=cesga/CN=johnsmith/VO=fedcloud.egi.eu/Role=NULL/Capability=NULL" --driver x509 | |||
<br> | |||
*And its properties: | *And its properties: | ||
$ oneuser update | |||
$ oneuser update <id_x509_user> | |||
X509_DN="/DC=es/DC=irisgrid/O=cesga/CN=johnsmith" | X509_DN="/DC=es/DC=irisgrid/O=cesga/CN=johnsmith" | ||
*Add OpenNebula x509 auth, modify /etc/one/auth/x509_auth.conf file: | *Add OpenNebula x509 auth, modify /etc/one/auth/x509_auth.conf file: | ||
# Path to the trusted CA directory. It should contain the trusted CA's for | # Path to the trusted CA directory. It should contain the trusted CA's for | ||
# the server, each CA certificate shoud be name CA_hash.0 | # the server, each CA certificate shoud be name CA_hash.0 | ||
:ca_dir: "/etc/grid-security/certificates" | :ca_dir: "/etc/grid-security/certificates" | ||
<br> For more information have a look at the OpenNebula Documentation on x509 Authentication: http://opennebula.org/documentation:archives:rel3.2:x509_auth | |||
==== OCCI client Usage ==== | |||
*Runnig on Ubuntu 12.04 | |||
$ curl -L https://get.rvm.io | sudo bash -s stable | $ curl -L https://get.rvm.io | sudo bash -s stable | ||
$ sudo usermod -a -G rvm | $ sudo usermod -a -G rvm <username> | ||
$ rvm user gemsets | $ rvm user gemsets | ||
$ rvm install ruby-1.9.3 | $ rvm install ruby-1.9.3 | ||
Line 336: | Line 358: | ||
$ gem install occi | $ gem install occi | ||
<br> | |||
*Configure your user cert: | *Configure your user cert: | ||
$ cat $HOME/.globus/usercert.pem $HOME/.globus/userkey.pem >> $HOME/.globus/usercred.pem | |||
<br> | |||
*That's it now we can use the new occi client: | *That's it now we can use the new occi client: | ||
<br> | |||
*To find out more about available options and defaults use | *To find out more about available options and defaults use | ||
$ occi --help | $ occi --help | ||
<br> | |||
*To run the client in an interactive mode use: | *To run the client in an interactive mode use: | ||
$ occi --interactive | $ occi --interactive | ||
$ occi --interactive --endpoint https://cloud.cesga.es:3202/ | $ occi --interactive --endpoint https://cloud.cesga.es:3202/ | ||
<br> | |||
*To list available resources use: | *To list available resources use: | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource compute --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource compute --auth x509 | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource storage --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource storage --auth x509 | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource network --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource network --auth x509 | ||
<br> | |||
*To describe available resources use | *To describe available resources use | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource compute --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource compute --auth x509 | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource storage --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource storage --auth x509 | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource network --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource network --auth x509 | ||
<br> | |||
*To describe specific resources use: | *To describe specific resources use: | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/compute/<OCCI_ID> --auth x509 | |||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/storage/<OCCI_ID> --auth x509 | |||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/network/<OCCI_ID> --auth x509 | |||
<br> | |||
*To list available OS templates or Resource templates use: | *To list available OS templates or Resource templates use: | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource os_tpl --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource os_tpl --auth x509 | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource resource_tpl --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource resource_tpl --auth x509 | ||
<br> | |||
*To describe a specific OS template or Resource template use | *To describe a specific OS template or Resource template use | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource os_tpl#debian6 --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource os_tpl#debian6 --auth x509 | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource resource_tpl#small --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource resource_tpl#small --auth x509 | ||
<br> | |||
*To create a compute resource with mixins use: | *To create a compute resource with mixins use: | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action create --resource compute --mixin os_tpl#debian6 --mixin resource_tpl#small --resource-title "My rOCCI VM" --auth x509 | $ occi --endpoint https://cloud.cesga.es:3202/ --action create --resource compute --mixin os_tpl#debian6 --mixin resource_tpl#small --resource-title "My rOCCI VM" --auth x509 | ||
<br> | |||
*To delete a compute resource use: | *To delete a compute resource use: | ||
$ occi --endpoint https://cloud.cesga.es:3202/ --action delete --resource https://cloud.cesga.es:3202/compute/<OCCI_ID> --auth x509 | |||
<br> | |||
{i} More info available [[https://github.com/gwdg/rOCCI/%7Chere]]. | |||
A beta version of the rOCCI client is available [[http://ibergrid.lip.pt/CLOUD/infrastructure/OpenNebula?action=AttachFile&do=get&target=occi-3.0.0.beta.0.gem | ==== OCCI+VOMS client usage ==== | ||
install it by running "gem install | |||
support will be a part of the next stable release. | A beta version of the rOCCI client is available [[http://ibergrid.lip.pt/CLOUD/infrastructure/OpenNebula?action=AttachFile&do=get&target=occi-3.0.0.beta.0.gem%7Chere]], you can install it by running "gem install <GEM_FILE>". The proxy/VOMS certificate support will be a part of the next stable release. | ||
*To test VOMS support yourselves, you can use the following. | *To test VOMS support yourselves, you can use the following. | ||
# voms-proxy-init -voms ops.vo.ibergrid.eu | # voms-proxy-init -voms ops.vo.ibergrid.eu | ||
<br> | |||
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource storage --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource storage --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | ||
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource network --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource network --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | ||
<br> | |||
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | ||
<br> | |||
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | ||
<br> | |||
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl#debian6 --action describe --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl#debian6 --action describe --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | ||
<br> | |||
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action create --resource-title "MyrOCCIVM" --mixin os_tpl#debian6 --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action create --resource-title "MyrOCCIVM" --mixin os_tpl#debian6 --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem | ||
<br> etc. Please, keep in mind that the rOCCI client is not yet finished. | |||
==== References ==== | |||
==== References ==== | |||
* https://github.com/gwdg/rOCCI-server#rocci-server---a-ruby-occi-server | * https://github.com/gwdg/rOCCI-server#rocci-server---a-ruby-occi-server | ||
* http://opennebula.org/documentation:archives:rel3.2:x509_auth | * http://opennebula.org/documentation:archives:rel3.2:x509_auth | ||
* https://github.com/gwdg/rOCCI | * https://github.com/gwdg/rOCCI |
Revision as of 15:09, 8 January 2013
OpenNebula
rOCCI
This section describes how to intall and configure rOCCI 0.5.x in SL6
Requirements
The following setup is recommended
- usage of the Ruby Version Manger (RVM)
- Ruby >= 1.9.3
- Bundler gem installed (use gem install bundler)
Installation
Apache and EMI2 middleware
- Install EMI2 gridsite packages:
# cd /etc/yum.repos.d # wget http://eticssoft.web.cern.ch/eticssoft/mock/emi-2-rc-sl6.repo # yum install canl-c gridsite gridsite-libs ca-policy-egi-core fetch-crl
- Install these packages:
# yum install httpd mod_ssl mod_proxy_html # chkconfig httpd on
RVM
- As oneadmin run :
$ curl -L https://get.rvm.io | sudo bash -s stable
- And now add oneadmin user to RVM group:
# usermod -a -G rvm oneadmin
- Now as oneadmin user:
$ source /etc/profile.d/rvm.sh $ rvm user gemsets
- Checkout latest version from GIT:
$ git clone git://github.com/gwdg/rOCCI-server.git
- Change to rOCCI folder:
$ cd rOCCI-server
- Use RVM:
$ rvm install ruby-1.9.3
In a different console install rvm deps if needed:
- Install dependencies before press 'q', from another console, for SL machine:
#yum install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion libexpat1-dev rubygem-xmlparser expat expat-devel
- Install gems and rocci service (v0.5.x in this case):
$ git checkout 0.5.x $ bundle install
Configuration
rOCCI-server comes with different backends. Check the etc folder for available backends (e.g. dummy, opennebula, ...). Each backend has an example configuration in a file with the name of the backend and the extension .json. Copy one of those files to etc/backend/default.json and adapt its content to your setting.
To configure the behaviour of compute, network and storage resource creation, edit the backend specific extensions of the OCCI model at etc/backend/$BACKEND/model (e.g. etc/backend/dummy/model for the dummy backend).
To change the predefined resource or OS templates, you can adapt the existing templates in etc/backend/$BACKEND/templates or add new templates. If resource or OS templates are already registered in the backend, they will be automatically discovered by rOCCI-server.
$ cd /var/lib/one/rOCCI-server/etc/backend/opennebula
- And now copy opennebula backend before edit it:
$ cp opennebula.json ../default.json $ cd ..
- And we change it to use our configuration
{ "resources":[ { "kind":"http://rocci.info/server/backend#opennebula", "attributes":{ "info":{ "rocci":{ "backend":{ "opennebula":{ "endpoint":"http://localhost:2633/RPC2", "admin":"rocci", "password":"*****", "scheme":"http://my.occi.service/" } } } }
} } ]
}
- Edit /var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates/compute.erb file to fit your cloud resources:
OpenNebula backend
If you want to change the actual deployment within OpenNebula you can change the OpenNebula templates in the files in /var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates.
To configure OpenNebula resource templates (e.g. small, medium, large, ...) change the files in etc/backend/opennebula/templates .
- As example to use x86_64 arch by default edit /var/lib/one/rOCCI-server/etc/backend/opennebula/templates and s change this line for these files (for small/medium/large):
... "architecture":{ "Mutable":true, "Required":false, "Type":"string", "Pattern":"x86|x64", "Default":"x64" ...
Add rocci user
The Sunstone, EC2 and OCCI services communicate with the core using a “server” user. OpenNebula creates the serveradmin account at bootstrap, with the authentication driver server_cipher (symmetric key).
This “server” user uses a special authentication mechanism that allows the servers to perform an operation on behalf of other user.
- To add rocci user we run as oneadmin:
$ oneuser create rocci <good_pass_here_from_default.json> --driver server_cipher $ oneuser chgrp rocci oneadmin
(!) User/Pass is the same that default.json conf file.
Apache conf
/!\ add apache user to cloud or oneadmin group:
# usermod -a -G cloud apache
Let passenger guide you through installing and or configuring Apache for you
- /!\ Add oneadmin to sudoers (/etc/sudoers) and for SL and RH flavors also set secure_path value, as example for ruby-1.9.3 for SL6:
.. Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327/bin:/ var/lib/one/.rvm/gems/ruby-1.9.3-p327@global/bin:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin:/usr/local/rvm/bin:/usr/lib64/qt-3.3/bin:/usr/local /rvm/gems/ruby-1.9.3-p327@rOCCI-server/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327/gems/passenger-3. 0.12/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/bin:/usr/bin/ruby:/usr/bin/gem:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin/ruby:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin/gem .. ..
- If you want to configure apache automatically, from /var/lib/one/rOCCI-server directory as oneadmin user run this ( {i} for SL6 installation these packages are required: apache2-prefork-dev libapr1-dev libaprutil1-dev)
$ gem install bundler $ bundle exec rvmsudo passenger-install-apache2-module
- This script shows this info at the end:
The Apache 2 module was successfully installed.
Please edit your Apache configuration file, and add these lines:
LoadModule passenger_module /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12/ext/apache2/mod_passenger.so PassengerRoot /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12 PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby
After you restart Apache, you are ready to deploy any number of Ruby on Rails applications on Apache, without any further Ruby on Rails-specific configuration!
(!) Now we must create /etc/httpd/conf.d/passenger.conf file
- as root: vi /etc/httpd/conf.d/passenger.conf to include these lines
# Passenger module for rOCCI LoadModule passenger_module /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12/ext/apache2/mod_passenger.so PassengerRoot /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12 PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby
- Manually check these files (In this case we will use 3202 port for rOCCI service):
edit /etc/httpd/conf.d/ssl.conf:
LoadModule ssl_module modules/mod_ssl.so LoadModule gridsite_module /usr/lib64/httpd/modules/mod_gridsite.so NameVirtualHost your.host.name:3202 Listen 443 Listen 3202 ..
Create a new VirtualHost in the conf.d directory of Apache (e.g. in /etc/httpd/conf.d/rocci.conf) with the following content:
<VirtualHost your.host.name:3202> ServerName your.host.name
SSLEngine on SSLProtocol all SSLCertificateFile /etc/grid-security/hostcert.pem SSLCertificateKeyFile /etc/grid-security/hostkey.pem
SSLProxyEngine on SSLCACertificatePath /etc/grid-security/certificates SSLCARevocationPath /etc/grid-security/certificates
SSLVerifyClient require SSLVerifyDepth 10 SSLOptions +StdEnvVars
# important, this needs to point to the public folder of your rOCCI-server DocumentRoot /var/lib/one/rOCCI-server/public <Directory /opt/rOCCI-server/public>
## This exports various bits of info into the CGI environment ## variables (and is needed for gridsite-admin.cgi to work.) GridSiteEnvs on
## Nice GridSite directory listings (without truncating file names!) GridSiteIndexes off
## If this is greater than zero, we will accept GSI Proxies for clients
## (full client certificates - eg inside web browsers - are always ok) GridSiteGSIProxyLimit 1
## This directive allows authorized people to write/delete files ## from non-browser clients - eg with htcp(1) GridSiteMethods ""
Allow from all Options -MultiViews </Directory>
LogLevel debug </VirtualHost>
/!\ Change /var/lib/one/rOCCI-server/public permissions as oneadmin:
$ chmod -R 770 /var/lib/one/rOCCI-server/public $ chmod 440 /etc/grid-security/hostkey.pem
/!\ Include apache user into cloud group:
# usermod -a -G cloud apache
/!\ You have to start/restart Apache before you can use rOCCI-server!
- Make sure that can validate fedcloud.egi.eu's certs, i.e. the following file exists:
# cat /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.egee.cesnet.cz.lsc /DC=org/DC=terena/DC=tcs/C=CZ/O=CESNET/CN=voms1.egee.cesnet.cz /C=NL/O=TERENA/CN=TERENA eScience SSL CA
- Finally:
/etc/init.d/httpd restart
rOCCI + VOMS
If you want to use X.509/VOMS authentication for your users, you need to create the users in OpenNebula with the X.509 driver. For a user named 'johnsmith' from fedcloud.egi.eu the command may look like this
$ oneuser create johnsmith "/DC=es/DC=irisgrid/O=cesga/CN=johnsmith|/DC=es/DC=irisgrid/O=cesga/CN=johnsmith/VO=fedcloud.egi.eu/Role=NULL/Capability=NULL" --driver x509
- And its properties:
$ oneuser update <id_x509_user> X509_DN="/DC=es/DC=irisgrid/O=cesga/CN=johnsmith"
- Add OpenNebula x509 auth, modify /etc/one/auth/x509_auth.conf file:
# Path to the trusted CA directory. It should contain the trusted CA's for # the server, each CA certificate shoud be name CA_hash.0 :ca_dir: "/etc/grid-security/certificates"
For more information have a look at the OpenNebula Documentation on x509 Authentication: http://opennebula.org/documentation:archives:rel3.2:x509_auth
OCCI client Usage
- Runnig on Ubuntu 12.04
$ curl -L https://get.rvm.io | sudo bash -s stable $ sudo usermod -a -G rvm <username> $ rvm user gemsets $ rvm install ruby-1.9.3 $ source /etc/profile.d/rvm.sh $ gem install occi
- Configure your user cert:
$ cat $HOME/.globus/usercert.pem $HOME/.globus/userkey.pem >> $HOME/.globus/usercred.pem
- That's it now we can use the new occi client:
- To find out more about available options and defaults use
$ occi --help
- To run the client in an interactive mode use:
$ occi --interactive $ occi --interactive --endpoint https://cloud.cesga.es:3202/
- To list available resources use:
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource compute --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource storage --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource network --auth x509
- To describe available resources use
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource compute --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource storage --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource network --auth x509
- To describe specific resources use:
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/compute/<OCCI_ID> --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/storage/<OCCI_ID> --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/network/<OCCI_ID> --auth x509
- To list available OS templates or Resource templates use:
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource os_tpl --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource resource_tpl --auth x509
- To describe a specific OS template or Resource template use
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource os_tpl#debian6 --auth x509 $ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource resource_tpl#small --auth x509
- To create a compute resource with mixins use:
$ occi --endpoint https://cloud.cesga.es:3202/ --action create --resource compute --mixin os_tpl#debian6 --mixin resource_tpl#small --resource-title "My rOCCI VM" --auth x509
- To delete a compute resource use:
$ occi --endpoint https://cloud.cesga.es:3202/ --action delete --resource https://cloud.cesga.es:3202/compute/<OCCI_ID> --auth x509
{i} More info available [[1]].
OCCI+VOMS client usage
A beta version of the rOCCI client is available [[2]], you can install it by running "gem install <GEM_FILE>". The proxy/VOMS certificate support will be a part of the next stable release.
- To test VOMS support yourselves, you can use the following.
# voms-proxy-init -voms ops.vo.ibergrid.eu
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource storage --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem # occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource network --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl#debian6 --action describe --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action create --resource-title "MyrOCCIVM" --mixin os_tpl#debian6 --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem
etc. Please, keep in mind that the rOCCI client is not yet finished.
References
* https://github.com/gwdg/rOCCI-server#rocci-server---a-ruby-occi-server * http://opennebula.org/documentation:archives:rel3.2:x509_auth * https://github.com/gwdg/rOCCI