Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "FAQ10 EGI Federated Cloud User"

From EGIWiki
Jump to navigation Jump to search
(Replaced content with "{{Template: Op menubar}} {{Template:Doc_menubar}} Category:Operations Manuals {{TOC_right}} {{Template:Block-comment | name=Warning | text=This FAQ is no longer ma...")
Tag: Replaced
 
(16 intermediate revisions by 6 users not shown)
Line 5: Line 5:
{{TOC_right}}  
{{TOC_right}}  


= EGI Federated Cloud User =
{{Template:Block-comment
 
| name=Warning
== Basics ==
| text=This FAQ is no longer maintained, please check the new documentation at https://egi-federated-cloud.readthedocs.io/en/latest/faq.html
 
  }}
=== How can I get access to the EGI.eu Federated Cloud? ===
See the [[Federated Cloud user support]] page for information.
 
=== How can I register to the FedCloud VO? ===
You can register to FedCloud VO at [https://perun.metacentrum.cz/perun-registrar-cert/?vo=fedcloud.egi.eu fedcloud.egi.eu Perun server]. Note that this VO is mainly for testing and demonstration purposes. For operational usage of the EGI Federated Cloud you will need to join an existing VO related to your own user community or create a new one. You can contact the [mailto:ucst@egi.eu EGI User Support Team] for more information.
 
=== How can I install the rOCCI client? ===
You can follow the [[Fedcloud-tf:CLI_Environment|CLI setup guide]] to install and setup the FedCloud User command line environment (which contains the rOCCI client).
 
=== How can I get a proxy certificate (for authorization)? ===
A VOMS proxy is obtained with the <code>voms-proxy-init</code> command. Please note that you need to be registered at the VO and that the proxy '''must''' have RFC format to work with all providers. You can use the following command line:
$ '''voms-proxy-init -voms fedcloud.egi.eu --rfc'''
 
== Site information ==
 
 
=== How can I get the list of the FedCloud sites? ===
The list of certified FedCloud sites is available [[Fedcloud-tf:ResourceProviders#Fully_integrated_Resource_Providers|in this page]].
 
The EGI Federated Cloud publishes also dynamic information about the sites (endpoints, available images, etc...) via the Federated Cloud information system, based on the [http://en.wikipedia.org/wiki/BDII BDII] technology. BDII has a standard LDAP interface, which you can query with any LDAP client. For example, on linux, you can execute the following commands to get the list of the EGI sites exposing respectively cloud IaaS resources (via OCCI) and cloud STaaS resources (via CDMI):
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=grid,o=glue "GLUE2ServiceType=IaaS" | grep GLUE2ServiceAdminDomainForeignKey
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=grid,o=glue "GLUE2ServiceType=STaaS" | grep GLUE2ServiceAdminDomainForeignKey
 
=== How can I get the FedCloud sites endpoints? ===
The endpoint information is available [[Federated_Cloud_user_support|here]], in the "Resource Endpoints" column of the table. Some site may expose only Infrastructure-as-a-Service (starting/stopping of VMs), which is served via the OCCI endpoint, while some other sites may expose only STorage-as-a-Service, served via the CDMI endpoint.
 
For the image management service, OCCI, it is also possible to get the list of the site supporting a given image with the endpoints and all the information about to start the image using the [https://appdb.egi.eu/browse/cloud EGI Application Database], browsing the list of images and checking the "Availability and Usage" page for the image.
 
It is also possible to query the site endpoints via the EGI Federated Cloud information management system (see [[#How can I get the list of the FedCloud sites?|How can I get the list of the FedCloud sites?]]), via the following LDAP queries (for respectively OCCI and CDMI endpoints):
 
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2DomainID=''<site_name>'',GLUE2GroupID=grid,o=glue GLUE2EndpointInterfaceName=OCCI | grep GLUE2EndpointURL
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2DomainID=''<site_name>'',GLUE2GroupID=grid,o=glue GLUE2EndpointInterfaceName=CDMI | grep GLUE2EndpointURL
 
=== How can I choose which FedCloud site to use? ===
FedCloud sites offer their resources voluntary to users through Virtual Organisations (VO). Before you can access a Federated Cloud site you need to join a Virtual Organisation supported by that site. The [[Federated_Cloud_user_support|Federated Cloud User Support page]] contains instructions on how to join an existing VO or how to establish a new VO.
 
=== How can I get the list of the resource templates available on a site? ===
In a command line environment, using the rOCCI client (a guide on how to install it is available [[Fedcloud-tf:CLI Environment|here]]), you can run the following command
 
occi --endpoint ''site_occi_endpoint'' --action list --resource resource_tpl --auth x509 --user-cred ''proxy_certificate'' --voms
 
where ''site_occi_endpoint'' is the site OCCI endpoint and ''proxy_certificate'' is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see [[Fedcloud-tf:CLI Environment#Create_a_proxy_certificate|here]])
 
If you need more information about the resource templates than the one available via the OCCI interface (eg. number of CPU available, quantity of memory available), you can use the EGI Information System (see [[#How can I get the list of the FedCloud sites?|How can I get the list of the FedCloud sites?]]), via the following LDAP query:
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=cloud,GLUE2DomainID=''<site_name>'',GLUE2GroupID=grid,o=glue objectClass=GLUE2ExecutionEnvironment
 
=== How can I get the list of the OS templates available on a site? ===
In a command line environment, using the rOCCI client (a guide on how to install it is available [[Fedcloud-tf:CLI Environment|here]]), you can run the following command
 
occi --endpoint ''site_occi_endpoint'' --action list --resource os_tpl --auth x509 --user-cred ''proxy_certificate'' --voms
 
where ''site_occi_endpoint'' is the site OCCI endpoint and ''proxy_certificate'' is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see [[Fedcloud-tf:CLI Environment#Create_a_proxy_certificate|here]])
 
If you need more information about the resource templates than the one available via the OCCI interface (eg. number of CPU available, quantity of memory available), you can use the EGI Information System (see [[#How can I get the list of the FedCloud sites?|How can I get the list of the FedCloud sites?]]), via the following LDAP query:
ldapsearch -x -H ldap://lcg-bdii.cern.ch:2170 -b GLUE2GroupID=cloud,GLUE2DomainID=''<site_name>'',GLUE2GroupID=grid,o=glue objectClass=GLUE2ApplicationEnvironment
 
== Managing VMs ==
 
=== How can I start a VM? ===
In a command line environment using the rOCCI client (Check the [[Fedcloud-tf:CLI Environment|guide for installing the CLI tools]]), you can run the following command
 
$ '''occi -e ''<occi_endpoint>'' --auth x509 --user-cred ''</path/to/proxy>'' --voms \'''
  '''-a create -r compute --mixin ''<os_template>'' --mixin ''<resource_template>'' --attribute occi.core.title="''<name_of_vm>''"'''
 
where <code>''<occi_endpoint>''</code> is the site OCCI endpoint, <code>''</path/to/proxy>'' is the path to your VOMS proxy certificate (obtained via the voms-proxy-init command), <code>''<os_template>''</code> is your VM OS image disk (e.x. os_tpl#debian6) and <code>''<resource_template>''</code> is your resource template (ex. resource_tpl#small).
 
The fastest way to obtain the <code>''<occi_endpoint>''</code>, <code>''<os_template>''</code> and <code>''<resource_template>''</code> parameters for any given site is to use [http://appdb.egi.eu/browse/cloud EGI Application Database].
 
=== How can I get the list of running VM? ===
In a command line environment, using the rOCCI client (a guide on how to install it is available [[Fedcloud-tf:CLI Environment|here]]), you can run the following command
 
occi --endpoint ''site_occi_endpoint'' --action list --resource compute --auth x509 --user-cred ''proxy_certificate'' --voms
 
where ''site_occi_endpoint'' is the site OCCI endpoint and ''proxy_certificate'' is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see [[Fedcloud-tf:CLI Environment#Create_a_proxy_certificate|here]])
 
=== How can I get the status of a VM? ===
 
In a command line environment using the rOCCI client (Check the [[Fedcloud-tf:CLI Environment|guide for installing the CLI tools]]), you can run the following command:
 
$ '''occi -e ''<occi_endpoint>'' --auth x509 --user-cred ''</path/to/proxy>'' --voms \'''
  '''-a describe -r /compute/''<VM_ID>''
 
where <code>''<occi_endpoint>''</code> is the site OCCI endpoint, <code>''</path/to/proxy>''</code> is the path to your VOMS proxy certificate, and <code>''<VM_ID>''</code> is the identifier of the VM as returned by the <code>create</code> action.
 
=== How can I connect to a VM? ===
 
As first, you need to check that the VM is running and get its IP address. You can do that via a ''describe'' OCCI command (see [[#How can I get the status of a VM?]] for more info).
 
To access the VM via the shell, you can use an SSH client and connect to the VM IP address, port 22.
 
NOTE: For all of the generic OS images and most of the custom images, no credentials are included within the image. The public key to access the VM is set up via contextualization (see [[#Contextualisation]])
 
=== How can I assign a public IP to a VM? ===
 
Some sites do not automatically assign a public IP address to a VM during the creation phase. In this case, you could need to attach it to the VM after the creation.
In a command line environment using the rOCCI client (Check the [[Fedcloud-tf:CLI Environment|guide for installing the CLI tools]]), you can run the following command:
 
$ '''occi -e ''<occi_endpoint>'' --auth x509 --user-cred ''</path/to/proxy>'' --voms \'''
  '''-a link -r /compute/''<VM_ID> --link /network/public''
 
where <code>''<occi_endpoint>''</code> is the site OCCI endpoint, <code>''</path/to/proxy>''</code> is the path to your VOMS proxy certificate, and <code>''<VM_ID>''</code> is the identifier of the VM as returned by the <code>create</code> action.
 
=== How can I delete a VM? ===
 
In a command line environment, using the rOCCI client (a guide on how to install it is available [[Fedcloud-tf:CLI Environment|here]]), you can run the following command
 
occi --endpoint ''site_occi_endpoint'' --action delete --resource /compute/''VMID'' --auth x509 --user-cred ''proxy_certificate'' --voms
 
where ''site_occi_endpoint'' is the site OCCI endpoint, ''proxy_certificate'' is the path to your VOMS proxy certificate (obtained via the voms-proxy-init commands, see [[Fedcloud-tf:CLI Environment#Create_a_proxy_certificate|here]]) and ''VMID'' is the VMID obtained as output of the create VM action. '''Once deleted, the VM cannot be recovered!'''
 
 
 
== Contextualisation ==
 
=== What is contextualisation? ===
 
The contextualisation of is the process of installing, configuring and preparing software upon boot time on a pre-defined virtual machine image. This way, the pre-defined images can be stored as generic and small as possible, since customisations will take place on boot time.
 
Contextualisation is particularly useful for:
* Configuration not known until instantiation (e.g. data location).
* Private Information (e.g. host certs)
* Software that changes frequently or under development.
 
The contextualisation requires passing some data to the VMs on instantiation (the context) and handling that context in the VM. EGI FedCloud has extended OCCI v.1.1 (current version) to support contextualisation via mixins.
 
=== How can I inject my public SSH key into the machine? ===
 
The best way to login into the virtual server is to use SSH keys. If you don't have one, you need to generate it with the <code>ssh-keygen</code> command:
ssh-keygen -f fedcloud
 
This will generate two files:
* <code>fedcloud</code>, the private key. This file should never be shared
* <code>fedcloud.pub</code>, the public key. That will be sent to your VM.
 
To inject the public SSH key into the VM you can use the <code>--context public_key="file:///<your public key location>"</code> option when creating a VM with occi-cli. cloud-init will take care of putting the key on the default configured user (depends on the VM).
 
You can also enforce the user to have a key with a contextualisation file:
<pre>
#cloud-config
  users:
    - name: cloudadm
      sudo: ALL=(ALL) NOPASSWD:ALL
      lock-passwd: true
      ssh-import-id: cloudadm
      ssh-authorized-keys:
        - <paste here the contents of your SSH key pub file>
</pre>
 
'''NOTE: the number of spaces at the beginning of each line in the contextualisation file must be respected in order to work'''
 
See below on how to use the contextualisation file.
 
=== How can I use contextualisation script? ===
If you have a contextualization script, you can use it with the <code>--context user_data=<context file></code> option as shown below:
 
$ '''occi -e ''<occi_endpoint>'' --auth x509 --user-cred ''</path/to/proxy>'' --voms \'''
  '''-action create -resource compute \'''
  '''--mixin ''<os_template>'' --mixin ''<resource_template>'' --attribute occi.core.title="''<name_of_vm>''"'''
  '''--context user_data="file:///''<path/to/contextualization_file>''"'''
 
You can use both <code>--context user_data</code> and <code>--context public_key</code> options at the same time.
'''NOTE: FedCloud supports [https://launchpad.net/cloud-init CloudInit] user data format. Using CloudInit you can easily inject your SSH private keys or run a set of deployment scripts. Read more at [http://cloudinit.readthedocs.org/ cloud-init documentation].'''
 
=== Contextualisation and configuration managers (e.g. Puppet, Chef, Ansible,.,.) ===
 
Contextualisation is not configuration management. Contextualisation can help you te start your server and do an initial configuration and is well suited for disposable nodes that can be easily thrown away (e.g. worker nodes). If you have a long running service or need to perform changes in the configuration during the lifetime of the machine, a configuration manager may be better suited. In that case, contextualisation can be used to bootstrap your configuration manager.
 
cloud-init supports the configuration of [https://www.chef.io/ chef] and [https://puppetlabs.com puppet] out of the box, check cloud-init documentation for:
* [http://cloudinit.readthedocs.org/en/latest/topics/examples.html#install-and-run-chef-recipes install and run chef recipes]
* [http://cloudinit.readthedocs.org/en/latest/topics/examples.html#setup-and-run-puppet setup and run puppet]
 
See also [http://stackoverflow.com/questions/11995290/why-do-people-use-puppet-chef-with-amazon-cloud-formation-instead-of-just-using this StackOverflow question on the matter].
 
== Images ==
=== How can I get information about the available VM images? ===
The [http://appdb.egi.eu/browse/cloud EGI Application Database] contains information about the VM images available in the EGI Federated Cloud. Within the [http://appdb.egi.eu/browse/cloud AppDB Cloud Marketplace], you can look for a VM and get all the information about which VO the VM is associated, the sites where the VM is available and the OCCI endpoint, resource template and OS template IDs to start the VMs. This information is present in the image "Availability and Usage" panel.
 
=== How can I publish my own VM image? ===
Direct publish of custom VMs implies an authorization and verification phase. Your VM will need to be approved by the VO administrator and added to the VO image list, for the EGI sites to download it and make it available to the VO members.
 
More information on the process to create a custom VM and publish it is reported [[HOWTO10|here]]
 
== How can I upload a VM image to the EGI FedCloud repository ==
The [https://appliance-repo.egi.eu/ EGI FedCloud appliance repository] is a common place to store the FedCloud images available to all the FedCloud members. You can optionally use your own storage for the VM image.
 
To be able to insert new images in the Appliance repository, you need to have an authorized user certificate, member of the fedloud.egi.eu voms organization. If you do not have an EGI certificate, you can request one following [http://www.egi.eu/how-to/get_a_certificate.html this guide]. If you gave a certificate but you are not a member of the fedloud.egi.eu VO, you can register here (https://perun.metacentrum.cz/perun-registrar-cert/?vo=fedcloud.egi.eu).
 
The steps to upload your image to the appliance repostory are:
 
1. Create the directory where you want to place your image:
  curl -k --cert usercert.pem --key userkey.pem -X PUT https://appliance-repo.egi.eu/images/base/SL-5.7-x86_64-base/
curl -k --cert usercert.pem --key userkey.pem -X PUT https://appliance-repo.egi.eu/images/base/SL-5.7-x86_64-base/1.0/
2. Upload the image:
curl -k --cert usercert.pem --key userkey.pem -T /path/to/image https://appliance-repo.egi.eu/images/base/SL-5.5-x86_64-base/1.0/SL-5.7-x86_64-base-1.0.ova
 
== Storage ==
 
=== The disk on my VM is full, how can I get more space? ===
There are several ways to increase the disk space available at the VM. The fastest and easiest one is to use block storage, creating a new storage disk device and attaching it to the VM. For more information about block storage you can refer to [[HOWTO09|this guide]]
 
=== How can I keep my data after the VM is stopped? ===
After a VM is stopped, the VM OS image disk is destroyed and all the data on top of it is lost. To save your data, you need to use some form of persistent storage, which can be block storage or object storage. According to your application needs, it may be better to use one or both the solutions. For more information about block storage and object storage in the EGI Federated Cloud you can refer to [[HOWTO09|this guide]]

Latest revision as of 15:39, 3 March 2020