Fedcloud-tf:WorkGroups:Scenario1:OpenNebulaInstallation

OpenNebula

rOCCI

This section describes how to intall and configure rOCCI 0.5.x in SL6

Requirements

The following setup is recommended

• usage of the Ruby Version Manger (RVM)
• Ruby >= 1.9.3
• Bundler gem installed (use gem install bundler)

Installation

Apache and EMI2 middleware

• Install EMI2 gridsite packages:

# cd /etc/yum.repos.d
# wget http://eticssoft.web.cern.ch/eticssoft/mock/emi-2-rc-sl6.repo
# yum install canl-c gridsite gridsite-libs ca-policy-egi-core fetch-crl

• Install these packages:

# yum install httpd mod_ssl mod_proxy_html
# chkconfig httpd on

RVM

• As oneadmin run :

$ curl -L https://get.rvm.io | sudo bash -s stable

• And now add oneadmin user to RVM group:

# usermod -a -G rvm oneadmin

• Now as oneadmin user:

$ source /etc/profile.d/rvm.sh
$ rvm user gemsets

• Checkout latest version from GIT:

$ git clone git://github.com/gwdg/rOCCI-server.git

• Change to rOCCI folder:

$ cd rOCCI-server

• Use RVM:

$ rvm install ruby-1.9.3

In a different console install rvm deps if needed:

• Install dependencies before press 'q', from another console, for SL machine:

#yum install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion libexpat1-dev rubygem-xmlparser expat expat-devel

• Install gems and rocci service (v0.5.x in this case):

$ git checkout 0.5.x
$ bundle install

Configuration

rOCCI-server comes with different backends. Check the etc folder for available backends (e.g. dummy, opennebula, ...). Each backend has an example configuration in a file with the name of the backend and the extension .json. Copy one of those files to etc/backend/default.json and adapt its content to your setting.

To configure the behaviour of compute, network and storage resource creation, edit the backend specific extensions of the OCCI model at etc/backend/$BACKEND/model (e.g. etc/backend/dummy/model for the dummy backend).

To change the predefined resource or OS templates, you can adapt the existing templates in etc/backend/$BACKEND/templates or add new templates. If resource or OS templates are already registered in the backend, they will be automatically discovered by rOCCI-server.

$ cd /var/lib/one/rOCCI-server/etc/backend/opennebula

• And now copy opennebula backend before edit it:

$ cp opennebula.json ../default.json
$ cd ..

• And we change it to use our configuration

{
  "resources":[
      {
          "kind":"http://rocci.info/server/backend#opennebula",
          "attributes":{
              "info":{
                  "rocci":{
                      "backend":{
                          "opennebula":{
                              "endpoint":"http://localhost:2633/RPC2",
                              "admin":"rocci",
                              "password":"*****",
                              "scheme":"http://my.occi.service/"
                          }
                      }
                  }
              }

           }
      }
  ]

}

• Edit /var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates/compute.erb file to fit your cloud resources:

OpenNebula backend

If you want to change the actual deployment within OpenNebula you can change the OpenNebula templates in the files in /var/lib/one/rOCCI-server/etc/backend/opennebula/one_templates.

To configure OpenNebula resource templates (e.g. small, medium, large, ...) change the files in etc/backend/opennebula/templates .

• As example to use x86_64 arch by default edit /var/lib/one/rOCCI-server/etc/backend/opennebula/templates and s change this line for these files (for small/medium/large):

...
                      "architecture":{
                          "Mutable":true,
                          "Required":false,
                          "Type":"string",
                          "Pattern":"x86|x64",
                          "Default":"x64"
...

Add rocci user

The Sunstone, EC2 and OCCI services communicate with the core using a “server” user. OpenNebula creates the serveradmin account at bootstrap, with the authentication driver server_cipher (symmetric key).

This “server” user uses a special authentication mechanism that allows the servers to perform an operation on behalf of other user.

• To add rocci user we run as oneadmin:

$ oneuser create rocci <good_pass_here_from_default.json> --driver server_cipher
$ oneuser chgrp rocci oneadmin

(!) User/Pass is the same that default.json conf file.

Apache conf

/!\ add apache user to cloud or oneadmin group:

# usermod -a -G cloud apache 

Let passenger guide you through installing and or configuring Apache for you

• /!\ Add oneadmin to sudoers (/etc/sudoers) and for SL and RH flavors also set secure_path value, as example for ruby-1.9.3 for SL6:

..
Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327/bin:/
var/lib/one/.rvm/gems/ruby-1.9.3-p327@global/bin:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin:/usr/local/rvm/bin:/usr/lib64/qt-3.3/bin:/usr/local
/rvm/gems/ruby-1.9.3-p327@rOCCI-server/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327/gems/passenger-3.
0.12/bin:/var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/bin:/usr/bin/ruby:/usr/bin/gem:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin/ruby:/usr/local/rvm/rubies/ruby-1.9.3-p327/bin/gem
..
..

• If you want to configure apache automatically, from /var/lib/one/rOCCI-server directory as oneadmin user run this ( {i} for SL6 installation these packages are required: apache2-prefork-dev libapr1-dev libaprutil1-dev)

$ gem install bundler
$ bundle exec rvmsudo passenger-install-apache2-module

• This script shows this info at the end:

The Apache 2 module was successfully installed.

Please edit your Apache configuration file, and add these lines:

  LoadModule passenger_module /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12/ext/apache2/mod_passenger.so
 PassengerRoot /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12
 PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby

After you restart Apache, you are ready to deploy any number of Ruby on Rails
applications on Apache, without any further Ruby on Rails-specific
configuration!

(!) Now we must create /etc/httpd/conf.d/passenger.conf file

• as root: vi /etc/httpd/conf.d/passenger.conf to include these lines

# Passenger module for rOCCI
LoadModule passenger_module /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12/ext/apache2/mod_passenger.so
PassengerRoot /var/lib/one/.rvm/gems/ruby-1.9.3-p327@rOCCI-server/gems/passenger-3.0.12
PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rOCCI-server/ruby

---

• Manually check these files (In this case we will use 3202 port for rOCCI service):

edit /etc/httpd/conf.d/ssl.conf:

LoadModule ssl_module modules/mod_ssl.so
LoadModule gridsite_module /usr/lib64/httpd/modules/mod_gridsite.so
NameVirtualHost your.host.name:3202
Listen 443
Listen 3202
..

Create a new VirtualHost in the conf.d directory of Apache (e.g. in /etc/httpd/conf.d/rocci.conf) with the following content:

<VirtualHost your.host.name:3202>
      ServerName your.host.name

       SSLEngine on
      SSLProtocol all
      SSLCertificateFile /etc/grid-security/hostcert.pem
      SSLCertificateKeyFile /etc/grid-security/hostkey.pem

       SSLProxyEngine on
      SSLCACertificatePath /etc/grid-security/certificates
      SSLCARevocationPath /etc/grid-security/certificates

       SSLVerifyClient require
      SSLVerifyDepth 10
      SSLOptions +StdEnvVars

       # important, this needs to point to the public folder of your rOCCI-server
      DocumentRoot /var/lib/one/rOCCI-server/public
<Directory /opt/rOCCI-server/public>

         ## This exports various bits of info into the CGI environment
        ## variables (and is needed for gridsite-admin.cgi to work.)
        GridSiteEnvs           on

         ## Nice GridSite directory listings (without truncating file names!)
        GridSiteIndexes off

         ## If this is greater than zero, we will accept GSI Proxies for clients

         ## (full client certificates - eg inside web browsers - are always ok)
        GridSiteGSIProxyLimit 1

         ## This directive allows authorized people to write/delete files
        ## from non-browser clients - eg with htcp(1)
        GridSiteMethods ""

         Allow from all
        Options -MultiViews
      </Directory>

       LogLevel debug
</VirtualHost>

/!\ Change /var/lib/one/rOCCI-server/public permissions as oneadmin:

$ chmod -R 770 /var/lib/one/rOCCI-server/public
$ chmod 440 /etc/grid-security/hostkey.pem

/!\ Include apache user into cloud group:

# usermod -a -G cloud apache

/!\ You have to start/restart Apache before you can use rOCCI-server!

• Make sure that can validate fedcloud.egi.eu's certs, i.e. the following file exists:

# cat /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.egee.cesnet.cz.lsc
/DC=org/DC=terena/DC=tcs/C=CZ/O=CESNET/CN=voms1.egee.cesnet.cz
/C=NL/O=TERENA/CN=TERENA eScience SSL CA

• Finally:

/etc/init.d/httpd restart

rOCCI + VOMS

If you want to use X.509/VOMS authentication for your users, you need to create the users in OpenNebula with the X.509 driver. For a user named 'johnsmith' from fedcloud.egi.eu the command may look like this

$ oneuser create johnsmith "/DC=es/DC=irisgrid/O=cesga/CN=johnsmith|/DC=es/DC=irisgrid/O=cesga/CN=johnsmith/VO=fedcloud.egi.eu/Role=NULL/Capability=NULL" --driver x509

• And its properties:

$ oneuser update <id_x509_user>
X509_DN="/DC=es/DC=irisgrid/O=cesga/CN=johnsmith"

• Add OpenNebula x509 auth, modify /etc/one/auth/x509_auth.conf file:

# Path to the trusted CA directory. It should contain the trusted CA's for
# the server, each CA certificate shoud be name CA_hash.0
:ca_dir: "/etc/grid-security/certificates"

For more information have a look at the OpenNebula Documentation on x509 Authentication: http://opennebula.org/documentation:archives:rel3.2:x509_auth

OCCI client Usage

• Runnig on Ubuntu 12.04

$ curl -L https://get.rvm.io | sudo bash -s stable
$ sudo usermod -a -G rvm <username>
$ rvm user gemsets
$ rvm install ruby-1.9.3
$ source /etc/profile.d/rvm.sh
$ gem install occi

• Configure your user cert:

$ cat $HOME/.globus/usercert.pem $HOME/.globus/userkey.pem >> $HOME/.globus/usercred.pem

• That's it now we can use the new occi client:

• To find out more about available options and defaults use

$ occi --help

• To run the client in an interactive mode use:

$ occi --interactive
$ occi --interactive --endpoint https://cloud.cesga.es:3202/

• To list available resources use:

$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource compute --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource storage --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource network --auth x509

• To describe available resources use

$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource compute --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource storage --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource network --auth x509

• To describe specific resources use:

$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/compute/<OCCI_ID> --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/storage/<OCCI_ID> --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource https://cloud.cesga.es:3202/network/<OCCI_ID> --auth x509

• To list available OS templates or Resource templates use:

$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource os_tpl --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action list --resource resource_tpl --auth x509

• To describe a specific OS template or Resource template use

$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource os_tpl#debian6 --auth x509
$ occi --endpoint https://cloud.cesga.es:3202/ --action describe --resource resource_tpl#small --auth x509

• To create a compute resource with mixins use:

$ occi --endpoint https://cloud.cesga.es:3202/ --action create --resource compute --mixin os_tpl#debian6 --mixin resource_tpl#small --resource-title "My rOCCI VM" --auth x509

• To delete a compute resource use:

$ occi --endpoint https://cloud.cesga.es:3202/ --action delete --resource https://cloud.cesga.es:3202/compute/<OCCI_ID> --auth x509

{i} More info available [[1]].

OCCI+VOMS client usage

A beta version of the rOCCI client is available [[2]], you can install it by running "gem install <GEM_FILE>". The proxy/VOMS certificate support will be a part of the next stable release.

• To test VOMS support yourselves, you can use the following.

# voms-proxy-init -voms ops.vo.ibergrid.eu

# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource storage --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem
# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource network --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem

# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem

# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl --action list --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem

# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource os_tpl#debian6 --action describe --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem

# occi --endpoint https://cloud.cesga.es:3202/ --auth x509 --resource compute --action create --resource-title "MyrOCCIVM" --mixin os_tpl#debian6 --user-cred /tmp/x509up_u1000 --proxy-ca ~/.globus/usercert.pem

etc. Please, keep in mind that the rOCCI client is not yet finished.

References

* https://github.com/gwdg/rOCCI-server#rocci-server---a-ruby-occi-server
* http://opennebula.org/documentation:archives:rel3.2:x509_auth
* https://github.com/gwdg/rOCCI