HOWTO11 How to use the rOCCI Client

From EGIWiki
(Redirected from HOWTO11)
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators


Contents

rOCCI client

rOCCI client provides a command line client that you can use directly from the shell to interact with OCCI endpoints of EGI FedCloud.

Installation

Pre-packaged VM

EGI provides a FedCloud clients Virtual Appliance in AppDB, this VA includes rOCCI-cli and voms-clients ready to use in EGI FedCloud. You can start use this VM locally (e.g. using VirtualBox), but you will need to pass some contextualisation data to be able to login. This is a 64bit OS image, in order to run in on VirtualBox on Windows, you must have Virtualization Technology enabled in your BIOS (may be disabled by default). The VM needs to be contextualised for local usage, and extended with your certificates (See subsections below).

Contextualise for local usage

EGI FedCloud clients VA is based on Ubuntu 14.04 and is ready to be contextualised with cloud-init. You can easily contextualise a VM in VirtualBox to set a password to the default user as described in cloud-init documentation. For convenience we have prepared a ISO file that you can add to your VM that sets as password passw0rd for the default user ubuntu. Follow these instructions to get your VM ready:

  1. Download the OVA file from https://appdb.egi.eu/store/vappliance/egi.fedcloud.clients
  2. Import the file into VirtualBox, by default it only uses 512MB of RAM, you may increase it for better performance
  3. Download the contextualisation iso
  4. Add the iso to your newly imported VM from the setting window of the VM (see screenshot below) as a new CD-ROM
  5. Start the VM and login as user ubuntu, password passw0rd
Add-context-disk.png

Copy certificates into VM

Once you are logged into the VM, you need to copy your certificate files to the /home/ubuntu/.globus directory so you can create a VOMS proxy. You can share a local folder of your computer with the VM and copy the files from there. To do so follow these steps:

  1. In the VM settings window, add a new folder in the Shared Folders tab.
  2. Select the folder on your computer where you have your certificates and set a name for it (e.g. certs)
  3. Login into your VM as user ubuntu
  4. Mount the folder on the VM with this command (change certs for the name you have set in the previous step):
    sudo mount -t vboxsf -o uid=$UID,gid=$(id -g) certs /mnt
  5. You now should be able to access the /mnt folder at your VM, you can copy your certificates to your home directory, check the Installation of certificate files below for more information on how to prepare your certificates.

Linux

EGI provides a script for installation of the OCCI CLI and VOMS with a single command. This script is known to work under Ubuntu 14, Ubuntu 12, RH6 derivatives (CentOS6, SL6). A known limitation: the command does not work with Java 1.8, but does work with Java 1.7.

Type this into your console to start the installation (with a user with sudo privileges):

curl -L http://go.egi.eu/fedcloud.ui | sudo /bin/bash -

or (directly as root):

curl -L http://go.egi.eu/fedcloud.ui | /bin/bash -


The script can also be used as user_data of a VM to contextualize a basic OS into a FedCloud client.

WARNING: if you experience authentication issues against a Resource Centre that is usually expected to be working (production-ready RCs), please take care that:

Mac OS X

Installation script also works under Mac OS X, although it has some prerequisites:

Once you have those installed, open a terminal and type:

curl -L http://go.egi.eu/fedcloud.ui | /bin/bash -

It will ask for your password to gain root privileges to create VOMS configuration files.

Docker

The egifedcloud/fedcloud-inserinterface docker image comes with OCCI-cli and fedcloud.egi.eu VO preconfigured. You can get it with:

docker pull egifedcloud/fedcloud-userinterface

To ease the usage of the docker client, you can get the git repository https://github.com/enolfc/fedcloud-userinterface where you can find one helper script: occi. This script will check if you have a valid proxy and create one for you if not found (expects to find certificates under ~/.globus, check #Installation_of_certificate_files for more information on certificates) and then runs the occi command against a endpoint defined in environment variable OCCI_ENDPOINT with any options passed, e.g.:

OCCI_ENDPOINT=http://server4-epsh.unizar.es:8787 ./occi --action list --resource compute

will execute action list on resource compute for endpoint http://server4-epsh.unizar.es:8787. $PWD/data will be mounted at the container when using this script.

Using Windows

In order to use the script on Windows follow this instructions (from the docker terminal):

  1. Follow the instructions below taking into account that in order to perform step 2 (copying the certificates to the machine) you can access your Windows home folder at /c/Users/<user name>/. For example of you have your YourCert.p12 file at your Desktop, you can use the following command (user name here is enol): cp /c/Users/enol/Desktop/YourCert.p12 ., all the other steps remain the same.
  2. Clone the git repository: git clone https://github.com/enolfc/fedcloud-userinterface.git
  3. cd into the git repo and start using the commands:
cd fedcloud-userinterface
OCCI_ENDPOINT=http://server4-epsh.unizar.es:8787 sh ./occi --action list --resource compute

Other Systems

rOCCI-cli is distributed as a Ruby gem, check rOCCI-cli github page for instructions. Packages for several Linux distributions are available at AppDB rOCCI CLI entry.

You will need to install also VOMS clients for your system in order to generate a proxy for accessing the EGI FedCloud resources. Follow VOMS documentation to install the software. Below you can find specific setup instructions for fedcloud.egi.eu:

mkdir -p /etc/grid-security/vomsdir/fedcloud.egi.eu

cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.grid.cesnet.cz.lsc << EOF 
/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms1.grid.cesnet.cz
/C=NL/ST=Noord-Holland/L=Amsterdam/O=TERENA/CN=TERENA eScience SSL CA 3
EOF

cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms2.grid.cesnet.cz.lsc << EOF 
/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms2.grid.cesnet.cz
/C=NL/ST=Noord-Holland/L=Amsterdam/O=TERENA/CN=TERENA eScience SSL CA 3
EOF

cat >> /etc/vomses << EOF 
"fedcloud.egi.eu" "voms1.grid.cesnet.cz" "15002" "/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms1.grid.cesnet.cz" "fedcloud.egi.eu" "24"
"fedcloud.egi.eu" "voms2.grid.cesnet.cz" "15002" "/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms2.grid.cesnet.cz" "fedcloud.egi.eu" "24"
EOF

Credentials

In order to use the Federated Cloud, you will need to join a Virtual Organisation (VO) of the cloud, and configure your rOCCI client to recognise this VO. The Federates Cloud User support page includes information on how to find a suitable cloud VO and on how to join it.

Configuration of VOs

If you have installed the rOCCI client with the scripts provided above OR if you are using the rOCCI client in the pre-installed virtual machine, the fedcloud.egi.eu VO is already preconfigured for you and you can jump to the next section. If you want to configure another VO, then read on.

A VO configuration needs two different pieces of information:

vomses files are stored by default at /etc/vomses/ and are normally named following this convention: <vo name>.<server name> (e.g. for fedcloud.egi.eu VO, you would have fedcloud.egi.eu.voms1.grid.cesnet.cz and fedcloud.egi.eu.voms2.grid.cesnet.cz. .lsc files are stored at /etc/grid-security/vomsdir/<vo name> and there should be one file for each of the VOMS server of the VO.

You can check the which are the specific VOMS servers for your VO at the Operations Portal. Normally each VOMS server has a Configuration Info link where the exact information to include in the vomses and .lsc files. For example for fedcloud.egi.eu, the voms2.grid.cesnet.cz, this information can be obtained at https://voms2.grid.cesnet.cz:8443/voms/fedcloud.egi.eu/configuration/configuration.action.

Installation of certificate files

Your certificate has to be installed in the user's home directory following these instructions:

1. Export or 'backup' the certificate from your web browser. The interface for this varies from browser to browser. The exported file will probably have the extension .p12 or .pfx. Guard this file carefully. Store it off your computer, or remove it once you are finished with this process.

2. Copy this file to the user's home directory where your rOCCI client is (This may require transfer of the file over the network.)

3. Create a directory in the user's home directory:

mkdir $HOME/.globus

4. Extract the certificate creating a public and a private key file replacing YourCert.p12 with the filename chosen during step 1:

openssl pkcs12 -in YourCert.p12 -clcerts -nokeys -out $HOME/.globus/usercert.pem
openssl pkcs12 -in YourCert.p12 -nocerts -out $HOME/.globus/userkey.pem

The user will be asked to define a passphrase during this step. This passphrase has to be entered every time a proxy is created from the certificate. For security reasons, an empty passphrase is not advisable. (and may even result a disfunctional user key.)

5. Set the access mode on your userkey.pem and usercert.pem files:

chmod 400 $HOME/.globus/userkey.pem 
chmod 600 $HOME/.globus/usercert.pem

6. Further protection of the $HOME/.globus directory is necessary to prevent everyone except the user to enter this directory:

chmod go-rx $HOME/.globus

If the $HOME/.globus directory holding the certificate resides in an afs home-directory, the directory has to be secured using afs-tools in addition to set the normal unix file access permissions:

fs setacl -dir $HOME/.globus -acl system:anyuser l

The user's certificate (usercert.pem and userkey.pem) can be copied to every other machine to access the cloud by transporting the $HOME/.globus directory. The security measures described above have to be repeated.

Proxy Generation

Clock of your machine needs to be synchronized, large clock skews will make the proxy generation fail!
CRLs should be up to date also, you can use the fetch-crl command to update them.


You can create a proxy with voms-proxy-init as follows, be sure to include the --rfc option:

$ voms-proxy-init -voms fedcloud.egi.eu --rfc
Enter GRID pass phrase for this identity:
Creating temporary proxy ............................................ Done
Contacting  voms2.grid.cesnet.cz:15002 [/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms2.grid.cesnet.cz] 
"fedcloud.egi.eu" Done
Creating proxy ........................................................................ Done

Your proxy is valid until Wed Aug  5 02:26:43 2015

X509_USER_PROXY variable

In the commands below, the X509_USER_PROXY variable is used to reference your proxy. You can define it with the following command (sh shells) if you have a working proxy:

$ export X509_USER_PROXY=`voms-proxy-info -path`

rOCCI-cli usage

How to find out more about available options and defaults use

occi --help

How to list the OS templates (aka VM Images) in a site

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource os_tpl
http://schemas.openstack.org/template/os#72ada03a-5694-4a79-8e7e-069516a31a59
http://schemas.openstack.org/template/os#d5b97735-747f-4f08-ab0b-1f84fe417714
http://schemas.openstack.org/template/os#ec4bb03e-d6df-4964-a490-ae0ef57536e7
http://schemas.openstack.org/template/os#d07a5f26-5f2e-453f-98f2-d0a8784ae980
http://schemas.openstack.org/template/os#f92e0a8b-1f34-424b-8682-1ba93799072d
http://schemas.openstack.org/template/os#39f2bed2-5a4f-419e-b4fb-33832e47d5af
http://schemas.openstack.org/template/os#f835d4a5-cb08-4350-b60e-3fdd0dc703a4
http://schemas.openstack.org/template/os#c0a2f9e0-081a-419c-b9a5-8cb03b1decb5
http://schemas.openstack.org/template/os#02f8cd09-7c79-4b3a-923a-51cd16496a6f
http://schemas.openstack.org/template/os#5364f77a-e1cb-4a6c-862e-96dc79c4ef67
http://schemas.openstack.org/template/os#7cfba655-f692-406f-a659-79b0224290cc
http://schemas.openstack.org/template/os#ff718bea-602b-4f13-91d2-58d134c45476
http://schemas.openstack.org/template/os#7664db29-e51f-4ab8-b4e7-3adfccee3150

How to get the description of an OS Template (VM Image)

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action describe --resource os_tpl#72ada03a-5694-4a79-8e7e-069516a31a59
#########################################################################################
[[ http://schemas.openstack.org/template/os#72ada03a-5694-4a79-8e7e-069516a31a59 ]]
title:        Image: Ubuntu-14.04-amd64
term:         72ada03a-5694-4a79-8e7e-069516a31a59
location:     /72ada03a-5694-4a79-8e7e-069516a31a59/
#########################################################################################

How to get the list of available resource templates (aka flavors)

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource resource_tpl
http://schemas.openstack.org/template/resource#m1-xlarge
http://schemas.openstack.org/template/resource#m1-large
http://schemas.openstack.org/template/resource#m1-tiny
http://schemas.openstack.org/template/resource#medium
http://schemas.openstack.org/template/resource#extra_large
http://schemas.openstack.org/template/resource#m1-small
http://schemas.openstack.org/template/resource#small
http://schemas.openstack.org/template/resource#large
http://schemas.openstack.org/template/resource#m1-medium

How to get the description of a resource template (aka flavor)

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action describe --resource resource_tpl#m1-xlarge
#########################################################################################
[[ http://schemas.openstack.org/template/resource#m1-xlarge ]]
title:        Flavor: m1.xlarge
term:         m1-xlarge
location:     /m1-xlarge/
#########################################################################################

How to create a key pair to access the VMs via SSH

In order to login into the server, you need to have a set of SSH keys. To generate a set of authentication keys, you can run:

ssh-keygen -t rsa -b 2048 -f tmpfedcloud

You can use the contextualisation to configure VMs with your SSH key, either with the public_key or user_data options.

cat > tmpfedcloud.login << EOF
#cloud-config
users:
  - name: cloudadm
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock-passwd: true
    ssh-import-id: cloudadm
    ssh-authorized-keys:
      - `cat tmpfedcloud.pub`
EOF

How to start a VM (create a compute resource)

$ OS_TPL=<paste here the OS Template (VM Image)>
$ RES_TPL=<paste here the resource template (flavor)>

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action create --resource compute --attribute occi.core.title="MyFirstVM" \
       --mixin $OS_TPL --mixin $RES_TPL \
       --context user_data="file://$PWD/tmpfedcloud.login"
https://some.cloud.site:6789/compute/86ae3606-d753-4421-b415-e697b1670879

How to get the description of a compute resource

$ occi --endpoint $ENDPOINT --voms --auth x509 --user-cred $X509_USER_PROXY \
       --action describe --resource $VM_ID
#########################################################################################
[[ http://schemas.ogf.org/occi/infrastructure#compute ]]
>> location: /compute/86ae3606-d753-4421-b415-e697b1670879
occi.core.id = 86ae3606-d753-4421-b415-e697b1670879
occi.compute.architecture = x86
occi.compute.cores = 2
occi.compute.hostname = myfirstvm
occi.compute.memory = 4.0
occi.compute.speed = 0.0
occi.compute.state = active
org.openstack.compute.state = active

Links:

    [[ http://schemas.ogf.org/occi/infrastructure#networkinterface ]]
    >> location: /network/interface/18047596-098d-4ce4-af75-ca7f908fbc09
    occi.networkinterface.gateway = 90.147.102.1
    occi.networkinterface.mac = fa:16:3e:c9:b1:a0
    occi.networkinterface.interface = eth0
    occi.networkinterface.state = active
    occi.networkinterface.allocation = static
    occi.networkinterface.address = 90.147.102.223
    occi.core.source = /compute/86ae3606-d753-4421-b415-e697b1670879
    occi.core.target = /network/admin
    occi.core.id = /network/interface/18047596-098d-4ce4-af75-ca7f908fbc09

Mixins:

    [[ http://schemas.openstack.org/compute/instance#os_vms ]]
    title:
    term:         os_vms
    location:     /os_vms/

    [[ http://schemas.openstack.org/template/os#72ada03a-5694-4a79-8e7e-069516a31a59 ]]
    title:        Image: Ubuntu-14.04-amd64
    term:         72ada03a-5694-4a79-8e7e-069516a31a59
    location:     /72ada03a-5694-4a79-8e7e-069516a31a59/

Actions:

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#stop ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#suspend ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#restart ]]
#########################################################################################

How to attach a public ip address to a compute resource

Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you may need to attach it to the VM after the creation.

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action link --resource $VM_ID --link /network/public

For some sites, you may need to replace the "/network/public" value by one chosen within the output of the following command:

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource network

This will list the available networks, from which you can find the one corresponding to the public network (where you get floating IP addresses in openstack language).

If the site is running OpenStack neutron, you may also need to specify the name of the pool where your floating IP address must be obtained from. Available pools are listed with the occi --dump-model command, they will be shown under the "@mixins" with the http://schemas.openstack.org/network/floatingippool scheme:

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms --dump-model \
          | grep "http://schemas.openstack.org/network/floatingippool"
external-net;scheme="http://schemas.openstack.org/network/floatingippool#";class="mixin";title="external-net";location="/mixin/external-net/"

Which then can be used in the link command as follows:

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
           --action link -r $VM_ID --link $NETWORK_ID \
           --mixin http://schemas.openstack.org/network/floatingippool#external-net

How to access the compute resource through SSH

$ ssh -i tmpfedcloud ubuntu@90.147.102.223
The authenticity of host '90.147.102.223 (90.147.102.223)' can't be established.
RSA key fingerprint is 31:76:97:09:d1:6b:3f:c0:21:02:36:9e:63:b5:f6:06.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '90.147.102.223' (RSA) to the list of known hosts.
Enter passphrase for key 'tmpfedcloud':
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Fri Jun 13 11:19:46 UTC 2014

  System load:  0.08              Processes:           79
  Usage of /:   55.4% of 1.32GB   Users logged in:     0
  Memory usage: 2%                IP address for eth0: 90.147.102.223
  Swap usage:   0%

  Graph this data and manage this system at:
    https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

$

How to delete the compute resource

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action delete --resource https://some.cloud.site:6789/compute/86ae3606-d753-4421-b415-e697b1670879

References

For more information, please, visit the rOCCI client GitHub repository

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Print/export