Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

FAQ10 EGI Federated Cloud User

From EGIWiki
Revision as of 10:14, 2 February 2016 by Vlegoll (talk | contribs) (misc small things)
Jump to navigation Jump to search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators


EGI Federated Cloud User

Basics

How can I get access to the EGI.eu Federated Cloud?

See the Federated Cloud user support page for information.

How can I register to the FedCloud VO?

You can register to FedCloud VO at fedcloud.egi.eu Perun server. Note that this VO is mainly for testing and demonstration purposes. For operational usage of the EGI Federated Cloud you will need to join an existing VO related to your own user community or create a new one. You can contact the EGI User Support Team for more information.

How can I install the rOCCI client?

You can follow the CLI setup guide to install and setup the FedCloud User command line environment (which contains the rOCCI client).

How can I get a proxy certificate (for authorization)?

A VOMS proxy is obtained with the voms-proxy-init command. Please note that you need to be registered at the VO and that the proxy must have RFC format to work with all providers. You can use the following command line:

$ voms-proxy-init -voms fedcloud.egi.eu --rfc

Site information

How can I get the list of the FedCloud sites?

The list of certified FedCloud sites is available in this page.

The EGI Federated Cloud publishes also dynamic information about the sites (endpoints, available images, etc...) via the Federated Cloud information system, based on the BDII technology. BDII has a standard LDAP interface, which you can query with any LDAP client. For example, on linux, you can execute the following commands to get the list of the EGI sites exposing respectively cloud IaaS resources (via OCCI) and cloud STaaS resources (via CDMI):

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=grid,o=glue "GLUE2ServiceType=IaaS" | grep GLUE2ServiceAdminDomainForeignKey
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=grid,o=glue "GLUE2ServiceType=STaaS" | grep GLUE2ServiceAdminDomainForeignKey

How can I get the FedCloud sites endpoints?

The endpoint information is available here, in the "Resource Endpoints" column of the table. Some site may expose only Infrastructure-as-a-Service (starting/stopping of VMs), which is served via the OCCI endpoint, while some other sites may expose only STorage-as-a-Service, served via the CDMI endpoint.

For the image management service, OCCI, it is also possible to get the list of the site supporting a given image with the endpoints and all the information about to start the image using the EGI Application Database, browsing the list of images and checking the "Availability and Usage" page for the image.

It is also possible to query the site endpoints via the EGI Federated Cloud information management system (see How can I get the list of the FedCloud sites?), via the following LDAP queries (for respectively OCCI and CDMI endpoints):

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue GLUE2EndpointInterfaceName=OCCI | grep GLUE2EndpointURL
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue GLUE2EndpointInterfaceName=CDMI | grep GLUE2EndpointURL

How can I choose which FedCloud site to use?

FedCloud sites offer their resources voluntary to users through Virtual Organisations (VO). Before you can access a Federated Cloud site you need to join a Virtual Organisation supported by that site. The Federated Cloud User Support page contains instructions on how to join an existing VO or how to establish a new VO.

AppDB shows the supported VOs and for each VO you can browse the resource providers that support it.

How can I get the list of the resource templates available on a site?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action list --resource resource_tpl --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint and proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here)

If you need more information about the resource templates than the one available via the OCCI interface (eg. number of CPU available, quantity of memory available), you can use the EGI Information System (see How can I get the list of the FedCloud sites?), via the following LDAP query:

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=cloud,GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue objectClass=GLUE2ExecutionEnvironment

How can I get the list of the OS templates available on a site?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action list --resource os_tpl --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint and proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here)

If you need more information about the resource templates than the one available via the OCCI interface (eg. number of CPU available, quantity of memory available), you can use the EGI Information System (see How can I get the list of the FedCloud sites?), via the following LDAP query:

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=cloud,GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue objectClass=GLUE2ApplicationEnvironment

Managing VMs

How can I start a VM?

In a command line environment using the rOCCI client (Check the guide for installing the CLI tools), you can run the following command

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a create -r compute --mixin <os_template> --mixin <resource_template> --attribute occi.core.title="<name_of_vm>"

where <occi_endpoint> is the site OCCI endpoint, </path/to/proxy> is the path to your VOMS proxy certificate (obtained via the voms-proxy-init command), <os_template> is your VM OS image disk (e.x. os_tpl#debian6) and <resource_template> is your resource template (ex. resource_tpl#small).

The fastest way to obtain the <occi_endpoint>, <os_template> and <resource_template> parameters for any given site is to use EGI Application Database.

How can I get the list of running VM?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action list --resource compute --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint and proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here)

How can I get the status of a VM?

In a command line environment using the rOCCI client (Check the guide for installing the CLI tools), you can run the following command:

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a describe -r /compute/<VM_ID>

where <occi_endpoint> is the site OCCI endpoint, </path/to/proxy> is the path to your VOMS proxy certificate, and <VM_ID> is the identifier of the VM as returned by the create action.

How can I connect to a VM?

As first, you need to check that the VM is running and get its IP address. You can do that via a describe OCCI command (see #How can I get the status of a VM? for more info).

To access the VM via the shell, you can use an SSH client and connect to the VM IP address, port 22.

NOTE: For all of the generic OS images and most of the custom images, no credentials are included within the image. The public key to access the VM is set up via contextualization (see #Contextualisation)

How can I assign a public IP to a VM?

Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you could need to attach it to the VM after the creation. In a command line environment using the rOCCI client (Check the guide for installing the CLI tools), you can run the following command:

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a link -r /compute/<VM_ID> --link /network/public

where <occi_endpoint> is the site OCCI endpoint, </path/to/proxy> is the path to your VOMS proxy certificate, and <VM_ID> is the identifier of the VM as returned by the create action.

How can I delete a VM?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action delete --resource /compute/VMID --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint, proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here) and VMID is the VMID obtained as output of the create VM action. Once deleted, the VM cannot be recovered!


Contextualisation

What is contextualisation?

The contextualisation of is the process of installing, configuring and preparing software upon boot time on a pre-defined virtual machine image. This way, the pre-defined images can be stored as generic and small as possible, since customisations will take place on boot time.

Contextualisation is particularly useful for:

  • Configuration not known until instantiation (e.g. data location).
  • Private Information (e.g. host certs)
  • Software that changes frequently or under development.

The contextualisation requires passing some data to the VMs on instantiation (the context) and handling that context in the VM. EGI FedCloud has extended OCCI v.1.1 (current version) to support contextualisation via mixins.

How can I inject my public SSH key into the machine?

The best way to login into the virtual server is to use SSH keys. If you don't have one, you need to generate it with the ssh-keygen command:

ssh-keygen -f fedcloud

This will generate two files:

  • fedcloud, the private key. This file should never be shared
  • fedcloud.pub, the public key. That will be sent to your VM.

To inject the public SSH key into the VM you can use the --context public_key="file:///<your public key location>" option when creating a VM with occi-cli. cloud-init will take care of putting the key on the default configured user (depends on the VM).

You can also enforce the user to have a key with a contextualisation file:

#cloud-config
  users:
    - name: cloudadm
      sudo: ALL=(ALL) NOPASSWD:ALL
      lock-passwd: true
      ssh-import-id: cloudadm
      ssh-authorized-keys:
        - <paste here the contents of your SSH key pub file>

NOTE: the number of spaces at the beginning of each line in the contextualisation file must be respected in order to work

See below on how to use the contextualisation file.

How can I use a contextualisation script?

If you have a contextualisation file, you can use it with the --context user_data=<context file> option as shown below:

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -action create -resource compute \
  --mixin <os_template> --mixin <resource_template> --attribute occi.core.title="<name_of_vm>"
  --context user_data="file:///<path/to/context_file>"

You can use both --context user_data and --context public_key options at the same time.

NOTE: FedCloud supports CloudInit user data format. Using CloudInit you can easily inject your SSH private keys or run a set of deployment scripts. Read more at cloud-init documentation.

Contextualisation and configuration managers (e.g. Puppet, Chef, Ansible,.,.)

Contextualisation is not configuration management. Contextualisation can help you start your server and do initial configuration. It is well suited for disposable nodes that can be easily thrown away (e.g. worker nodes). If you have a long running service or need to perform changes in the configuration during the lifetime of the machine, a configuration manager may be better suited. In that case, contextualisation can be used to bootstrap your configuration manager.

cloud-init supports the configuration of chef and puppet out of the box, check cloud-init documentation for:

See also this StackOverflow question on the matter.

How can I pass secrets to my VMs?

If you use a https endpoint for creating your VMs, information passed to contextualise the VMs can be assumed to be safe and only readable within your VM. However, take into account that anyone with access to the VM may be able to access also the contextualisation information.

Images

How can I get information about the available VM images?

The EGI Application Database contains information about the VM images available in the EGI Federated Cloud. Within the AppDB Cloud Marketplace, you can look for a VM and get all the information about which VO the VM is associated, the sites where the VM is available and the OCCI endpoint, resource template and OS template IDs to start the VMs. This information is present in the image "Availability and Usage" panel.

How can I publish my own VM image?

Direct publish of custom VMs implies an authorization and verification phase. Your VM will need to be approved by the VO administrator and added to the VO image list, for the EGI sites to download it and make it available to the VO members.

More information on the process to create a custom VM and publish it is reported here

How can I upload a VM image to the EGI FedCloud repository

The EGI FedCloud appliance repository is a common place to store the FedCloud images available to all the FedCloud members. You can optionally use your own storage for the VM image.

To be able to insert new images in the Appliance repository, you need to have an authorized user certificate, member of the fedloud.egi.eu voms organization. If you do not have an EGI certificate, you can request one following this guide. If you gave a certificate but you are not a member of the fedloud.egi.eu VO, you can register here (https://perun.metacentrum.cz/perun-registrar-cert/?vo=fedcloud.egi.eu).

The steps to upload your image to the appliance repostory are:

1. Create the directory where you want to place your image:

curl -k --cert usercert.pem --key userkey.pem -X PUT https://appliance-repo.egi.eu/images/base/SL-5.7-x86_64-base/
curl -k --cert usercert.pem --key userkey.pem -X PUT https://appliance-repo.egi.eu/images/base/SL-5.7-x86_64-base/1.0/

2. Upload the image:

curl -k --cert usercert.pem --key userkey.pem -T /path/to/image https://appliance-repo.egi.eu/images/base/SL-5.5-x86_64-base/1.0/SL-5.7-x86_64-base-1.0.ova

Storage

The disk on my VM is full, how can I get more space?

There are several ways to increase the disk space available at the VM. The fastest and easiest one is to use block storage, creating a new storage disk device and attaching it to the VM. For more information about block storage you can refer to this guide

How can I keep my data after the VM is stopped?

After a VM is stopped, the VM OS image disk is destroyed and all the data on top of it is lost. To save your data, you need to use some form of persistent storage, which can be block storage or object storage. According to your application needs, it may be better to use one or both the solutions. For more information about block storage and object storage in the EGI Federated Cloud you can refer to this guide