Difference between revisions of "HOWTO11 How to use the rOCCI Client"

From EGIWiki
Jump to: navigation, search
m (How to attach a public ip address to a compute resource)
m (How to attach a public ip address to a compute resource)
Line 328: Line 328:
 
== How to attach a public ip address to a compute resource ==
 
== How to attach a public ip address to a compute resource ==
 
Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you could need to attach it to the VM after the creation.
 
Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you could need to attach it to the VM after the creation.
 
You can list the available networks to find which one correspond to the public network (where you get floating IP addresses in openstack language)
 
  
 
<pre>
 
<pre>
 
$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
 
$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource network
+
       --action link --resource $VM_ID --link /network/public
 
</pre>
 
</pre>
  
And use the returned value instead of "/network/public" for the --link parameter value in the following command:
+
For some sites, you may need to replace the "/network/public" value by one from the output of the following command:
  
 
<pre>
 
<pre>
 
$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
 
$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action link --resource $VM_ID --link /network/public
+
       --action list --resource network
 
</pre>
 
</pre>
 +
 +
This will list the available networks, and then you can find which one correspond to the public network (where you get floating IP addresses in openstack language).
  
 
== How to access the compute resource through SSH ==
 
== How to access the compute resource through SSH ==

Revision as of 14:34, 12 April 2016

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators


rOCCI client

rOCCI client provides a command line client that you can use directly from the shell to interact with OCCI endpoints of EGI FedCloud.

Installation

Pre-packaged VM

EGI provides a FedCloud clients Virtual Appliance in AppDB, this VA includes rOCCI-cli and voms-clients ready to use in EGI FedCloud. You can start use this VM locally (e.g. using VirtualBox), but you will need to pass some contextualisation data to be able to login. This is a 64bit OS image, in order to run in on VirtualBox on Windows, you must have Virtualization Technology enabled in your BIOS (may be disabled by default).


Contextualise for local usage

EGI FedCloud clients VA is based on Ubuntu 14.04 and is ready to be contextualised with cloud-init. You can easily contextualise a VM in VirtualBox to set a password to the default user as described in cloud-init documentation. For convenience we have prepared a ISO file that you can add to your VM that sets as password passw0rd for the default user ubuntu. Follow these instructions to get your VM ready:

  1. Download the OVA file from https://appdb.egi.eu/store/vappliance/egi.fedcloud.clients
  2. Import the file into VirtualBox, by default it only uses 512MB of RAM, you may increase it for better performance
  3. Download the contextualisation iso
  4. Add the iso to your newly imported VM from the setting window of the VM (see screenshot below) as a new CD-ROM
  5. Start the VM and login as user ubuntu, password passw0rd
Add-context-disk.png

Linux

EGI provides a script for installation of the OCCI CLI and VOMS with a single command. This script is known to work under Ubuntu 14, Ubuntu 12, RH6 derivatives (CentOS6, SL6). A known limitation: the command does not work with Java 1.8, but does work with Java 1.7.

Type this into your console to start the installation (with a user with sudo privileges):

curl -L http://go.egi.eu/fedcloud.ui | sudo /bin/bash -

or (directly as root):

curl -L http://go.egi.eu/fedcloud.ui | /bin/bash -


The script can also be used as user_data of a VM to contextualize a basic OS into a FedCloud client.

Mac OS X

Installation script also works under Mac OS X, although it has some prerequisites:

  • Homebrew installed (used for installing fetch-crl and voms-clients)
  • Working Ruby>=1.9.3 installation, if not available, check RVM for installation

Once you have those installed, open a terminal and type:

curl -L http://go.egi.eu/fedcloud.ui | /bin/bash -

It will ask for your password to gain root privileges to create VOMS configuration files.

Docker

The egifedcloud/fedcloud-inserinterface docker image comes with OCCI-cli and fedcloud.egi.eu VO preconfigured. You can get it with:

docker pull egifedcloud/fedcloud-userinterface

To ease the usage of the docker client, you can get the git repository https://github.com/enolfc/fedcloud-userinterface where you can find one helper script: occi. This script will check if you have a valid proxy and create one for you if not found (expects to find certificates under ~/.globus, check #Installation_of_certificate_files for more information on certificates) and then runs the occi command against a endpoint defined in environment variable OCCI_ENDPOINT with any options passed, e.g.:

OCCI_ENDPOINT=http://server4-epsh.unizar.es:8787 ./occi --action list --resource compute

will execute action list on resource compute for endpoint http://server4-epsh.unizar.es:8787. $PWD/data will be mounted at the container when using this script.

Using Windows

In order to use the script on Windows follow this instructions (from the docker terminal):

  1. Follow the instructions below taking into account that in order to perform step 2 (copying the certificates to the machine) you can access your Windows home folder at /c/Users/<user name>/. For example of you have your YourCert.p12 file at your Desktop, you can use the following command (user name here is enol): cp /c/Users/enol/Desktop/YourCert.p12 ., all the other steps remain the same.
  2. Clone the git repository: git clone https://github.com/enolfc/fedcloud-userinterface.git
  3. cd into the git repo and start using the commands:
cd fedcloud-userinterface
OCCI_ENDPOINT=http://server4-epsh.unizar.es:8787 sh ./occi --action list --resource compute

Other Systems

rOCCI-cli is distributed as a Ruby gem, check rOCCI-cli github page for instructions. Packages for several Linux distributions are available at AppDB rOCCI CLI entry.

You will need to install also VOMS clients for your system in order to generate a proxy for accessing the EGI FedCloud resources. Follow VOMS documentation to install the software. Below you can find specific setup instructions for fedcloud.egi.eu:

mkdir -p /etc/grid-security/vomsdir/fedcloud.egi.eu

cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.grid.cesnet.cz.lsc << EOF 
/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms1.grid.cesnet.cz
/C=NL/ST=Noord-Holland/L=Amsterdam/O=TERENA/CN=TERENA eScience SSL CA 3
EOF

cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms2.grid.cesnet.cz << EOF 
/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms2.grid.cesnet.cz
/C=NL/ST=Noord-Holland/L=Amsterdam/O=TERENA/CN=TERENA eScience SSL CA 3
EOF

cat >> /etc/vomses << EOF 
"fedcloud.egi.eu" "voms1.grid.cesnet.cz" "15002" "/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms1.grid.cesnet.cz" "fedcloud.egi.eu" "24"
"fedcloud.egi.eu" "voms2.grid.cesnet.cz" "15002" "/DC=org/DC=terena/DC=tcs/C=CZ/ST=Hlavni mesto Praha/L=Praha 6/O=CESNET/CN=voms2.grid.cesnet.cz" "fedcloud.egi.eu" "24"
EOF

Credentials

In order to use the Federated Cloud, you will need to join a Virtual Organisation (VO) of the cloud, and configure your rOCCI client to recognise this VO. The Federates Cloud User support page includes information on how to find a suitable cloud VO and on how to join it.

Configuration of VOs

If you have installed the rOCCI client with the scripts provided above OR if you are using the rOCCI client in the pre-installed virtual machine, the fedcloud.egi.eu VO is already preconfigured for you and you can jump to the next section. If you want to configure another VO, then read on.

A VO configuration needs two different pieces of information:

  • the vomses configuration files, where the details of the VO are stored (e.g. name, server, ports)
  • the .lsc files that describe the trust chain of the VOMS server.

vomses files are stored by default at /etc/vomses/ and are normally named following this convention: <vo name>.<server name> (e.g. for fedcloud.egi.eu VO, you would have fedcloud.egi.eu.voms1.grid.cesnet.cz and fedcloud.egi.eu.voms2.grid.cesnet.cz. .lsc files are stored at /etc/grid-security/vomsdir/<vo name> and there should be one file for each of the VOMS server of the VO.

You can check the which are the specific VOMS servers for your VO at the Operations Portal. Normally each VOMS server has a Configuration Info link where the exact information to include in the vomses and .lsc files. For example for fedcloud.egi.eu, the voms2.grid.cesnet.cz, this information can be obtained at https://voms2.grid.cesnet.cz:8443/voms/fedcloud.egi.eu/configuration/configuration.action.

Installation of certificate files

Your certificate has to be installed in the user's home directory following these instructions:

1. Export or 'backup' the certificate from your web browser. The interface for this varies from browser to browser. The exported file will probably have the extension .p12 or .pfx. Guard this file carefully. Store it off your computer, or remove it once you are finished with this process.

2. Copy this file to the user's home directory where your rOCCI client is (This may require transfer of the file over the network.)

3. Create a directory in the user's home directory:

mkdir $HOME/.globus

4. Extract the certificate creating a public and a private key file replacing YourCert.p12 with the filename chosen during step 1:

openssl pkcs12 -in YourCert.p12 -clcerts -nokeys -out $HOME/.globus/usercert.pem
openssl pkcs12 -in YourCert.p12 -nocerts -out $HOME/.globus/userkey.pem

The user will be asked to define a passphrase during this step. This passphrase has to be entered every time a proxy is created from the certificate. For security reasons, an empty passphrase is not advisable. (and may even result a disfunctional user key.)

5. Set the access mode on your userkey.pem and usercert.pem files:

chmod 400 $HOME/.globus/userkey.pem 
chmod 600 $HOME/.globus/usercert.pem

6. Further protection of the $HOME/.globus directory is necessary to prevent everyone except the user to enter this directory:

chmod go-rx $HOME/.globus

If the $HOME/.globus directory holding the certificate resides in an afs home-directory, the directory has to be secured using afs-tools in addition to set the normal unix file access permissions:

fs setacl -dir $HOME/.globus -acl system:anyuser l

The user's certificate (usercert.pem and userkey.pem) can be copied to every other machine to access the cloud by transporting the $HOME/.globus directory. The security measures described above have to be repeated.

Proxy Generation

You can create a proxy with voms-proxy-init as follows, be sure to include the --rfc option:

$ voms-proxy-init -voms fedcloud.egi.eu --rfc
Enter GRID pass phrase for this identity:
Creating temporary proxy ............................................ Done
Contacting  voms2.grid.cesnet.cz:15002 [/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms2.grid.cesnet.cz] 
"fedcloud.egi.eu" Done
Creating proxy ........................................................................ Done

Your proxy is valid until Wed Aug  5 02:26:43 2015

X509_USER_PROXY variable

In the commands below, the X509_USER_PROXY variable is used to reference your proxy. You can define it with the following command (sh shells) if you have a working proxy:

$ export X509_USER_PROXY=\`voms-proxy-info -path\`

rOCCI-cli usage

How to find out more about available options and defaults use

occi --help


How to list the OS templates (aka VM Images) in a site

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource os_tpl
http://schemas.openstack.org/template/os#72ada03a-5694-4a79-8e7e-069516a31a59
http://schemas.openstack.org/template/os#d5b97735-747f-4f08-ab0b-1f84fe417714
http://schemas.openstack.org/template/os#ec4bb03e-d6df-4964-a490-ae0ef57536e7
http://schemas.openstack.org/template/os#d07a5f26-5f2e-453f-98f2-d0a8784ae980
http://schemas.openstack.org/template/os#f92e0a8b-1f34-424b-8682-1ba93799072d
http://schemas.openstack.org/template/os#39f2bed2-5a4f-419e-b4fb-33832e47d5af
http://schemas.openstack.org/template/os#f835d4a5-cb08-4350-b60e-3fdd0dc703a4
http://schemas.openstack.org/template/os#c0a2f9e0-081a-419c-b9a5-8cb03b1decb5
http://schemas.openstack.org/template/os#02f8cd09-7c79-4b3a-923a-51cd16496a6f
http://schemas.openstack.org/template/os#5364f77a-e1cb-4a6c-862e-96dc79c4ef67
http://schemas.openstack.org/template/os#7cfba655-f692-406f-a659-79b0224290cc
http://schemas.openstack.org/template/os#ff718bea-602b-4f13-91d2-58d134c45476
http://schemas.openstack.org/template/os#7664db29-e51f-4ab8-b4e7-3adfccee3150

How to get the description of an OS Template (VM Image)

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action describe --resource os_tpl#72ada03a-5694-4a79-8e7e-069516a31a59
#########################################################################################
[[ http://schemas.openstack.org/template/os#72ada03a-5694-4a79-8e7e-069516a31a59 ]]
title:        Image: Ubuntu-14.04-amd64
term:         72ada03a-5694-4a79-8e7e-069516a31a59
location:     /72ada03a-5694-4a79-8e7e-069516a31a59/
#########################################################################################

How to get the list of available resource templates (aka flavors)

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource resource_tpl
http://schemas.openstack.org/template/resource#m1-xlarge
http://schemas.openstack.org/template/resource#m1-large
http://schemas.openstack.org/template/resource#m1-tiny
http://schemas.openstack.org/template/resource#medium
http://schemas.openstack.org/template/resource#extra_large
http://schemas.openstack.org/template/resource#m1-small
http://schemas.openstack.org/template/resource#small
http://schemas.openstack.org/template/resource#large
http://schemas.openstack.org/template/resource#m1-medium

How to get the description of a resource template (aka flavor)

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action describe --resource resource_tpl#m1-xlarge
#########################################################################################
[[ http://schemas.openstack.org/template/resource#m1-xlarge ]]
title:        Flavor: m1.xlarge
term:         m1-xlarge
location:     /m1-xlarge/
#########################################################################################

How to create a key pair to access the VMs via SSH

In order to login into the server, you need to have a set of SSH keys. To generate a set of authentication keys, you can run:

ssh-keygen -t rsa -b 2048 -f tmpfedcloud

You can use the contextualisation to configure VMs with your SSH key, either with the public_key or user_data options.

  • public_key: just add the -T public_key=file:///path/of/public/key using the appropriate path to the public key file (it will be named tmpfedcloud.pub if you used the command above. This will add the key to the default user of the VM image (e.g. ubuntu for Ubuntu based distributions, centos for CentOS based)
  • user_data: cloud-init is also able to specify keys for specific users with a contextualisation script, that can be then used with the -T user_data=file:///path/to/contextfile. You can use the following command to create such script:
cat > tmpfedcloud.login << EOF
#cloud-config
users:
  - name: cloudadm
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock-passwd: true
    ssh-import-id: cloudadm
    ssh-authorized-keys:
      - `cat tmpfedcloud.pub`
EOF

How to start a VM (create a compute resource)

$ OS_TPL=<paste here the OS Template (VM Image)>
$ RES_TPL=<paste here the resource template (flavor)>

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action create --resource compute --attribute occi.core.title="MyFirstVM" \
       --mixin $OS_TPL --mixin $RES_TPL \
       --context user_data="file://$PWD/tmpfedcloud.login"
https://some.cloud.site:6789/compute/86ae3606-d753-4421-b415-e697b1670879

How to get the description of a compute resource

$ occi --endpoint $ENDPOINT --voms --auth x509 --user-cred $X509_USER_PROXY \
       --action describe --resource $VM_ID
#########################################################################################
[[ http://schemas.ogf.org/occi/infrastructure#compute ]]
>> location: /compute/86ae3606-d753-4421-b415-e697b1670879
occi.core.id = 86ae3606-d753-4421-b415-e697b1670879
occi.compute.architecture = x86
occi.compute.cores = 2
occi.compute.hostname = myfirstvm
occi.compute.memory = 4.0
occi.compute.speed = 0.0
occi.compute.state = active
org.openstack.compute.state = active

Links:

    [[ http://schemas.ogf.org/occi/infrastructure#networkinterface ]]
    >> location: /network/interface/18047596-098d-4ce4-af75-ca7f908fbc09
    occi.networkinterface.gateway = 90.147.102.1
    occi.networkinterface.mac = fa:16:3e:c9:b1:a0
    occi.networkinterface.interface = eth0
    occi.networkinterface.state = active
    occi.networkinterface.allocation = static
    occi.networkinterface.address = 90.147.102.223
    occi.core.source = /compute/86ae3606-d753-4421-b415-e697b1670879
    occi.core.target = /network/admin
    occi.core.id = /network/interface/18047596-098d-4ce4-af75-ca7f908fbc09

Mixins:

    [[ http://schemas.openstack.org/compute/instance#os_vms ]]
    title:
    term:         os_vms
    location:     /os_vms/

    [[ http://schemas.openstack.org/template/os#72ada03a-5694-4a79-8e7e-069516a31a59 ]]
    title:        Image: Ubuntu-14.04-amd64
    term:         72ada03a-5694-4a79-8e7e-069516a31a59
    location:     /72ada03a-5694-4a79-8e7e-069516a31a59/

Actions:

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#stop ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#suspend ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#restart ]]
#########################################################################################

How to attach a public ip address to a compute resource

Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you could need to attach it to the VM after the creation.

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action link --resource $VM_ID --link /network/public

For some sites, you may need to replace the "/network/public" value by one from the output of the following command:

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action list --resource network

This will list the available networks, and then you can find which one correspond to the public network (where you get floating IP addresses in openstack language).

How to access the compute resource through SSH

$ ssh -i tmpfedcloud cloudadm@90.147.102.223
The authenticity of host '90.147.102.223 (90.147.102.223)' can't be established.
RSA key fingerprint is 31:76:97:09:d1:6b:3f:c0:21:02:36:9e:63:b5:f6:06.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '90.147.102.223' (RSA) to the list of known hosts.
Enter passphrase for key 'tmpfedcloud':
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Fri Jun 13 11:19:46 UTC 2014

  System load:  0.08              Processes:           79
  Usage of /:   55.4% of 1.32GB   Users logged in:     0
  Memory usage: 2%                IP address for eth0: 90.147.102.223
  Swap usage:   0%

  Graph this data and manage this system at:
    https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

$

How to delete the compute resource

$ occi --endpoint $ENDPOINT --auth x509 --user-cred $X509_USER_PROXY --voms \
       --action delete --resource https://some.cloud.site:6789/compute/86ae3606-d753-4421-b415-e697b1670879

References

For more information, please, visit the rOCCI client GitHub repository