Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

FAQ10 EGI Federated Cloud User

From EGIWiki
Jump to navigation Jump to search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators


EGI Federated Cloud User

Basics

How can I get access to the EGI.eu Federated Cloud?

See the Federated Cloud user support page for information.

How can I register to the FedCloud VO?

You can register to FedCloud VO at fedcloud.egi.eu Perun server. Note that this VO is mainly for testing and demonstration purposes. For operational usage of the EGI Federated Cloud you will need to join an existing VO related to your own user community or create a new one. You can contact the EGI User Support Team for more information.

How can I install the rOCCI client?

You can follow the CLI setup guide to install and setup the FedCloud User command line environment (which contains the rOCCI client).

How can I get a proxy certificate (for authorization)?

A VOMS proxy is obtained with the voms-proxy-init command. Please note that you need to be registered at the VO and that the proxy must have RFC format to work with all providers. You can use the following command line:

$ voms-proxy-init -voms fedcloud.egi.eu --rfc

Site information

How can I get the list of the FedCloud sites?

The list of certified FedCloud sites is available in this page.

The EGI Federated Cloud publishes also dynamic information about the sites (endpoints, available images, etc...) via the Federated Cloud information system, based on the BDII technology. BDII has a standard LDAP interface, which you can query with any LDAP client. For example, on linux, you can execute the following commands to get the list of the EGI sites exposing respectively cloud IaaS resources (via OCCI) and cloud STaaS resources (via CDMI):

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=grid,o=glue "GLUE2ServiceType=IaaS" | grep GLUE2ServiceAdminDomainForeignKey
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=grid,o=glue "GLUE2ServiceType=STaaS" | grep GLUE2ServiceAdminDomainForeignKey

How can I get the FedCloud sites endpoints?

The endpoint information is available here, in the "Resource Endpoints" column of the table. Some site may expose only Infrastructure-as-a-Service (starting/stopping of VMs), which is served via the OCCI endpoint, while some other sites may expose only STorage-as-a-Service, served via the CDMI endpoint.

For the image management service, OCCI, it is also possible to get the list of the site supporting a given image with the endpoints and all the information about to start the image using the EGI Application Database, browsing the list of images and checking the "Availability and Usage" page for the image.

It is also possible to query the site endpoints via the EGI Federated Cloud information management system (see How can I get the list of the FedCloud sites?), via the following LDAP queries (for respectively OCCI and CDMI endpoints):

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue GLUE2EndpointInterfaceName=OCCI | grep GLUE2EndpointURL
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue GLUE2EndpointInterfaceName=CDMI | grep GLUE2EndpointURL

How can I choose which FedCloud site to use?

FedCloud sites offer their resources voluntary to users through Virtual Organisations (VO). Before you can access a Federated Cloud site you need to join a Virtual Organisation supported by that site. The Federated Cloud User Support page contains instructions on how to join an existing VO or how to establish a new VO.

How can I get the list of the resource templates available on a site?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action list --resource resource_tpl --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint and proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here)

If you need more information about the resource templates than the one available via the OCCI interface (eg. number of CPU available, quantity of memory available), you can use the EGI Information System (see How can I get the list of the FedCloud sites?), via the following LDAP query:

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=cloud,GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue objectClass=GLUE2ExecutionEnvironment

How can I get the list of the OS templates available on a site?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action list --resource os_tpl --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint and proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here)

If you need more information about the resource templates than the one available via the OCCI interface (eg. number of CPU available, quantity of memory available), you can use the EGI Information System (see How can I get the list of the FedCloud sites?), via the following LDAP query:

ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=cloud,GLUE2DomainID=<site_name>,GLUE2GroupID=grid,o=glue objectClass=GLUE2ApplicationEnvironment

Managing VMs

How can I start a VM?

In a command line environment using the rOCCI client (Check the guide for installing the CLI tools), you can run the following command

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a create -r compute --mixin <os_template> --mixin <resource_template> --attribute occi.core.title="<name_of_vm>"

where <occi_endpoint> is the site OCCI endpoint, </path/to/proxy> is the path to your VOMS proxy certificate (obtained via the voms-proxy-init command), <os_template> is your VM OS image disk (e.x. os_tpl#debian6) and <resource_template> is your resource template (ex. resource_tpl#small).

The fastest way to obtain the <occi_endpoint>, <os_template> and <resource_template> parameters for any given site is to use EGI Application Database.

How can I get the list of running VM?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action list --resource compute --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint and proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here)

How can I get the status of a VM?

In a command line environment using the rOCCI client (Check the guide for installing the CLI tools), you can run the following command:

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a describe -r /compute/<VM_ID>

where <occi_endpoint> is the site OCCI endpoint, </path/to/proxy> is the path to your VOMS proxy certificate, and <VM_ID> is the identifier of the VM as returned by the create action.

How can I connect to a VM?

As first, you need to check that the VM is running and get its IP address. You can do that via a describe OCCI command (see #How can I get the status of a VM? for more info).

To access the VM via the shell, you can use an SSH client and connect to the VM IP address, port 22.

NOTE: For all of the generic OS images and most of the custom images, no credentials are included within the image. The public key to access the VM is set up via contextualization (see [#Contextualisation])

How can I assign a public IP to a VM?

Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you could need to attach it to the VM after the creation. In a command line environment using the rOCCI client (Check the guide for installing the CLI tools), you can run the following command:

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a link -r /compute/<VM_ID> --link /network/public

where <occi_endpoint> is the site OCCI endpoint, </path/to/proxy> is the path to your VOMS proxy certificate, and <VM_ID> is the identifier of the VM as returned by the create action.

How can I delete a VM?

In a command line environment, using the rOCCI client (a guide on how to install it is available here), you can run the following command

occi --endpoint site_occi_endpoint --action delete --resource /compute/VMID --auth x509 --user-cred proxy_certificate --voms

where site_occi_endpoint is the site OCCI endpoint, proxy_certificate is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see here) and VMID is the VMID obtained as output of the create VM action. Once deleted, the VM cannot be recovered!


Contextualisation

What is contextualisation?

The contextualisation of is the process of installing, configuring and preparing software upon boot time on a pre-defined virtual machine image. This way, the pre-defined images can be stored as generic and small as possible, since customisations will take place on boot time.

Contextualisation is particularly useful for:

  • Configuration not known until instantiation (e.g. data location).
  • Private Information (e.g. host certs)
  • Software that changes frequently or under development.

The contextualisation requires passing some data to the VMs on instantiation (the context) and handling that context in the VM. EGI FedCloud has extended OCCI v.1.1 (current version) to support contextualisation via mixins.

How can I inject my public SSH key into the machine?

Best way to login into the virtual server is to use SSH keys. To inject your public SSH key into the VM during startup, you can use CloudInit contextualization feature and contextualization (see #How can I use contextualization?). A sample user data script to setup an administrator user is

#cloud-config
users:
 - name: cloudadm
   sudo: ALL=(ALL) NOPASSWD:ALL
   lock-passwd: true
   ssh-import-id: cloudadm
   ssh-authorized-keys:
     - <your SSH key>

NOTE: the number of spaces at the beginning of each line in the contextualization file must be respected in order to work

How can I use contextualization?

If you have a contextualization script, you can run it using the following rOCCI command at VM creation:

$ occi -e <occi_endpoint> --auth x509 --user-cred </path/to/proxy> --voms \
  -a create -r compute --mixin <os_template> --mixin <resource_template> --attribute occi.core.title="<name_of_vm>"
  -T user_data="file:///path/to/contextualization_script.sh"

NOTE: FedCloud supports CloudInit user data format. Using CloudInit you can easily inject your SSH private keys or run a set of deployment scripts. Read more at cloud-init documentation.

Contextualisation and configuration managers (e.g. Puppet, Chef, Ansible,.,.)

Contextualisation is not configuration management. Contextualisation can help you te start your server and do an initial configuration and is well suited for disposable nodes that can be easily thrown away (e.g. worker nodes). If you have a long running service or need to perform changes in the configuration during the lifetime of the machine, a configuration manager may be better suited. In that case, contextualisation can be used to bootstrap your configuration manager.

cloud-init supports the configuration of chef and puppet out of the box, check cloud-init documentation for:

See also this StackOverflow question on the matter.

Images

How can I get information about the available VM images?

The EGI Application Database contains information about the VM images available in the EGI Federated Cloud. Within the AppDB Cloud Marketplace, you can look for a VM and get all the information about which VO the VM is associated, the sites where the VM is available and the OCCI endpoint, resource template and OS template IDs to start the VMs. This information is present in the image "Availability and Usage" panel.

How can I publish my own VM image?

Direct publish of custom VMs implies an authorization and verification phase. Your VM will need to be approved by the VO administrator and added to the VO image list, for the EGI sites to download it and make it available to the VO members.

More information on the process to create a custom VM and publish it is reported here

How can I upload a VM image to the EGI FedCloud repository

The EGI FedCloud appliance repository is a common place to store the FedCloud images available to all the FedCloud members. You can optionally use your own storage for the VM image.

To be able to insert new images in the Appliance repository, you need to have an authorized user certificate, member of the fedloud.egi.eu voms organization. If you do not have an EGI certificate, you can request one following this guide. If you gave a certificate but you are not a member of the fedloud.egi.eu VO, you can register here (https://perun.metacentrum.cz/perun-registrar-cert/?vo=fedcloud.egi.eu).

The steps to upload your image to the appliance repostory are:

1. Create the directory where you want to place your image:

curl -k --cert usercert.pem --key userkey.pem -X PUT https://appliance-repo.egi.eu/images/base/SL-5.7-x86_64-base/
curl -k --cert usercert.pem --key userkey.pem -X PUT https://appliance-repo.egi.eu/images/base/SL-5.7-x86_64-base/1.0/

2. Upload the image:

curl -k --cert usercert.pem --key userkey.pem -T /path/to/image https://appliance-repo.egi.eu/images/base/SL-5.5-x86_64-base/1.0/SL-5.7-x86_64-base-1.0.ova

Storage

The disk on my VM is full, how can I get more space?

There are several ways to increase the disk space available at the VM. The fastest and easiest one is to use block storage, creating a new storage disk device and attaching it to the VM. For more information about block storage you can refer to this guide

How can I keep my data after the VM is stopped?

After a VM is stopped, the VM OS image disk is destroyed and all the data on top of it is lost. To save your data, you need to use some form of persistent storage, which can be block storage or object storage. According to your application needs, it may be better to use one or both the solutions. For more information about block storage and object storage in the EGI Federated Cloud you can refer to this guide