Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "HOWTO15 How to configure the Federated Cloud BDII"

From EGIWiki
Jump to navigation Jump to search
 
(37 intermediate revisions by 7 users not shown)
Line 1: Line 1:
[[Category: Technology ]]
{{Template:Op menubar}} {{Template:Doc_menubar}}
[[Category: Fedcloud-tf]]
[[Category:Deprecated]]
= Purpose =
{| style="border:1px solid black; background-color:lightgrey; color: black; padding:5px; font-size:140%; width: 90%; margin: auto;"
This page provides instructions on how to configure the Federated Cloud Production BDII.
| style="padding-right: 15px; padding-left: 15px;" |
|[[File:Alert.png]] This page is '''Deprecated'''; the content has been moved to https://docs.egi.eu/providers/cloud-compute/openstack/#egi-information-system 
|}
 
{{TOC_right}}
 
= Purpose =
 
This page provides instructions on how to configure the Federated Cloud Production resource-BDII.  
 
= Installation  =
 
Documentation and general installation guidelines for the cloud-info-provider are available at https://github.com/EGI-FCTF/cloud-bdii-provider
 
If you don't have already a BDII-site service in production, you will need to install also the <code>bdii</code> package (available in EPEL repo).
 
If you are using OpenStack as provider, you also need to install the <code>python-novaclient</code> package.
 
If you want to install from packages, here follow some guidelines.
 
== RHEL/CentOS/ScientificLinux  ==
 
*Add EPEL repository according to the instructions at https://fedoraproject.org/wiki/EPEL
*Add the cloud-info-provider repository to yum:
 
wget http://repository.egi.eu/community/software/cloud.info.provider/0.x/releases/repofiles/sl-6-x86_64.repo \
    -O /etc/yum.repos.d/cloud-info-provider.repo
 
*Install the package
 
yum install cloud-info-provider-service
 
== For Debian/Ubuntu 6/7/8  ==
 
*Add AppDB's repository:


= Configuration guide =
sudo wget http://repository.egi.eu/community/software/cloud.info.provider/0.x/releases/repofiles/ubuntu-precise-amd64.list \
        -O /etc/apt/sources.list.d/cloud-info-provider.list


== For sites with existing production BDII ==
*Add AppDB key:
If your site has already a production BDII (eg. for existing Grid resources), you need to add to this production BDII the cloud information. This information is in a separated dedicated three, so it will not interfere with your existing data. The production BDII at your site may be implemented using the site BDII software package released via UMD/EMI or with a bare OpenLDAP server. To add the production cloud information, you need to:


=== If your site has a BDII from EMI/UMD ===
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E2E992EB352D3E14
# Be sure that your site BDII supports GLUE2.0 schema and publish all the user relevant information on GLUE2.0 . Be sure also that the GOCDB GIIS entry of your production site points to the GLUE2.0 schema, (eg: <code>ldap://prisma-cloud.ba.infn.it:2170/GLUE2DomainID=PRISMA-INFN-BARI,o=glue </code>). If that is not the case, update your BDII to GLUE2.0 and update the GIIS URL accordingly.
# Download and configure the Python script in a directory of your choice, as described into the [[#Generate the cloud information|following chapter]]. Set the <code>provider['full_bdii_schema']</code> variable to 'false' .
# Run the Python script to produce the cloud LDIF
# Delete old cloud data (if any)(eg. <code> ldapdelete -H ldap://full.hostname:2170 'GLUE2GroupID=cloud,GLUE2DomainID=SZTAKI,o=glue' -D 'your LDAP admin DN' -W </code>)
# Add new cloud data (eg. <code>ldapadd -f cloud-ldif.ldif -H ldap://full.hostname:2170 -D 'your LDAP admin DN' -W</code>)
# Check that the cloud data has been successfully added without errors (eg. <code>ldapsearch -x -H ldap://full.hostname:2170 -b 'GLUE2GroupID=cloud,GLUE2DomainID=SZTAKI,o=glue'</code>)
# Move the cloud generation script or link it into the local BDII_PROVIDER_DIR (as default: <code>/var/lib/bdii/gip/provider/</code>. The BDII_PROVIDER_DIR folder path is set into <code>/etc/bdii/bdii.conf</code>)


=== If you site has a bare OpenLDAP server ===
*Install the package
# Be sure that your LDAP supports GLUE2.0 schema and publish all the user relevant information on GLUE2.0 . Be sure also that the GOCDB GIIS entry of your production site points to the GLUE2.0 schema, (eg: <code>ldap://prisma-cloud.ba.infn.it:2170/GLUE2DomainID=PRISMA-INFN-BARI,o=glue </code>). If that is not the case, update your BDII to GLUE2.0 and update the GIIS URL accordingly.
# Run the Python script to produce the cloud LDIF, setting the <code>provider['full_bdii_schema']</code> variable to 'false' . More information on how to run the schema is in the [[#Generate the cloud information|following chapter]].
# Delete old cloud data (if any)(eg. <code> $ ldapdelete -H ldap://full.hostname:2170 'GLUE2GroupID=cloud,GLUE2DomainID=SZTAKI,o=glue' -D 'your LDAP admin DN' -W </code>)
# Add new cloud data (eg. <code>ldapadd -f cloud-ldif.ldif -H ldap://full.hostname:2170 -D 'your LDAP admin DN' -W</code>)


== For sites without existing production BDII ==
sudo apt-get update
If your site does not have a site BDII, you can setup a new one simply by setting up a new LDAP server. To do so:
sudo apt-get install python-cloud-info-provider


# Install OpenLDAP packages (or any LDAP server you like) and configure it using the following configuration files:
= Configuration  =
#* Configuration file for the LDAP server. [https://raw.githubusercontent.com/EGI-FCTF/BDIIscripts/master/old/slapd.conf (slapd.conf) Download here].
#* GLUE20.schema file, containing the GLUE20 LDAP definitions. [https://raw.githubusercontent.com/EGI-FCTF/BDIIscripts/master/old/GLUE20.schema (GLUE20.schema) download here].
# Follow the [[#Generate the cloud information|Generate the cloud information]] section to produce the cloud LDIF, setting the <code>provider['full_bdii_schema']</code> variable to 'true' . More information on how to run the schema is in the following chapter
# Delete existing LDAP database directory, as specified in the slapd.conf file (if not already empty) (eg: <code>rm -rf /var/db/openldap/openldap-data/</code>) and create a new empty one (eg: <code>mkdir /var/db/openldap/openldap-data/; chown ldap:ldap -R /var/db/openldap/openldap-data/</code>)
# Load the ldap data (eg: <code> $ slapadd -f /etc/openldap/slapd.conf -l cloud-ldif.ldif </code>)
# Start the LDAP server (eg: <code>$ slapd -f /etc/openldap/slapd.conf -h ldap://full.hostname:2170 -d 0  > file-name 2>&1 &</code> or <code>service slapd start</code>)
# Configure your [https://goc.egi.eu/ GOCDB] site information the 'GIIS URL' with the address of your site BDII and the base schema, (eg: <code> ldap://prisma-cloud.ba.infn.it:2170/GLUE2DomainID=PRISMA-INFN-BARI,o=glue </code>)


= Refresh the data =
Use one of the template files in <code>/etc/cloud-info-provider</code> as basis for creating your own YAML file with the static information of your resources. E.g:
Data contained in the LDAP server is static, thus refresh or update need to be performed by hand. If you need to perform major updates, you can re-generate the entire cloud schema (using the Data publish script, as reported in the chapter below) and then re-import it. To do it, you can follow again the instruction in the [[#Configuration_Guide|Configuration Guide]] above.


= Generate the cloud information =
cp /etc/cloud-info-provider/sample.openstack.yaml /etc/cloud-info-provider/bdii.yaml
Cloud information can be generated using the following Python script ([https://raw.githubusercontent.com/EGI-FCTF/BDIIscripts/master/old/cloud-info-provider-site-entry-glue2 (cloud-info-provider-site-entry-glue2) Download here])


The script must be edited with your site and cloud services information before its execution, as described in the following tables. To run the script, just execute:
Each middleware has its own options to fetch the dynamic information, check the <code>--help</code> option for more information. Some additional notes are given in the following sections.  
  $ chmod +x cloud-info-provider-site-entry-glue2
  $ ./cloud-info-provider-site-entry-glue2 > cloud-ldif.ldif


===General interface information===
== General ==
{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black;"
 
!Variable name
*Site name will be fetched from <code>site</code> -&gt; <code>name</code> in the template file. Set it to the name defined in GOCDB. Alternatively, the site name can be fetched from /etc/glite-info-static/site/site.cfg (or by the file set with the --glite-site-info-static option)
!Description
 
!Examples
== OpenNebula ==
|-
 
|interface['IaaS_api']
*Use the <code>--middleware opennebularocci</code> option to activate this provider
|API standard for the IaaS management interface
 
|OCCI
*Use the <code>sample.opennebularocci.yaml</code> configuration sample. Change the following lines:
|-
<pre>
|interface['IaaS_api_version']
site:
|API standard version for the IaaS management interface
    # Your site name, as in GODCB (if omitted or set to None, this value is
|1.1
    # retreived from /etc/glite-info-static/site/site.cfg )
|-
    #name: SITE_NAME
|interface['IaaS_api_endpoint_technology']
</pre>
|API endpoint technology for the IaaS management interface
<pre>
|REST
compute:
|-
    # Total number of cores available
|interface['IaaS_api_authorization_method']
    total_cores: 0
|API AAI for the IaaS management interface
    # Total RAM available (GB)
|X509-VOMS
    total_ram: 0
|-
    # Hypervisor name (e.g. KVM, Xen, etc.)
|interface['STaaS_api']
    hypervisor: Foo Hypervisor
|API standard for the STaaS management interface
    # Hypervisor version
|CDMI
    hypervisor_version: 0.0.0
|-
    # ...
|interface['STaaS_api_version']
    # Middleware version
|API standard version for the STaaS management interface
    middleware_version: 0.0
|1.0.1
</pre>
|-
<pre>
|interface['STaaS_api_endpoint_technolog']
    endpoints:
|API endpoint technology for the STaaS management interface
        # ...
|REST
 
|-
        # Host serving your Virtual Machine Management interface (rOCCI-server)
|interface['STaaS_api_authorization_method']
        https://cloud-service01.example.org:11443:
|API AAI for the STaaS management interface
            endpoint_url: https://cloud-service01.example.org:11443
|X509-VOMS
</pre>
|}
<pre>
    # Images are retreived automatically by the endpoint
    images:
        defaults:
            #...
            # os_tpl schema prefix must match ROCCI_SERVER_OPENNEBULA_SCHEMA_NAMESPACE in rOCCI-server, only the host part
            schema: http://schemas.cloud-service01.example.org/occi/infrastructure/os_tpl
</pre>
 
*You need to specify connection parameters to the OpenNebula XML-RPC interface:
**<code>--on-auth</code> parameter should contain the authorization parameters for an existing user with full read permissions on the image disks. If the user has been created with the <code>core</code> driver, this parameter shall be set to <code>&lt;username&gt;:&lt;password&gt;</code>.
**<code>--on-rpcxml-endpoint</code> shall contain the address of the RPCv2 endpoint. Usually it is <code><nowiki>http://<hostname>:2633/RPC2</nowiki></code>. If not on a secure network, it is suggested to provide this interface via https, since the <code>on-auth</code> parameter will be sent in clear text to the server.
 
*Compute templates can be gathered in two ways. One option does not preclude the other and the resulting templates will be the merge of the two.
**directly from remote OpenNebula (with rOCCI-server installed and configured), by setting the <code>--rocci-remote-templates</code> flag
**manually by placing them in the configuration file
 
*With the <code>--cloudkeeper-images</code> flag, OS templates can be filtered so only the cloudkeeper ones are published.
 
== OpenStack  ==
 
*Use the <code>--middleware openstack</code> option to activate this provider
 
*The OpenStack provider uses python-novaclient (needs to be installed separately)
**<code>--os-username</code>, <code>--os-password</code>, <code>--auth-tenant-name</code>, <code>--os-auth-url</code>, <code>--os-cacert</code>, <code>--insecure</code> options to the cloud-provider allow to set the connection parameters. Alternatively you can use environment variables (e.g. <code>OS_USERNAME</code>) as with other OpenStack clients
**<code>--insecure</code> should not be used in production!
 
*Be sure that keystone contains the OCCI endpoint, otherwise it will not be published by the BDII. You can check this via the command <code>keystone service-list</code>. To create a new service and endpoint, you can run
<pre>keystone service-create --name occi_api --type occi --description 'Nova OCCI Service'
 
keystone endpoint-create --service_id &lt;service-id&gt; --region RegionOne --publicurl http://$HOSTNAME:8787/ \
                                        --internalurl http://$HOSTNAME:8787/ --adminurl http://$HOSTNAME:8787/
</pre>
where the ''service-id'' is the one obtained from <code>keystone service-list</code>.
 
*By default, the provider script will filter images without marketplace uri defined into the marketplace or vmcatcher_event_ad_mpuri property. If you want to list all the images templates (included local snapshots), set the variable 'require_marketplace_id: false' under 'compute' -&gt; 'images' -&gt; 'defaults' in the YAML configuration file.
 
== Create the provider  ==
 
*Create the file <code>/var/lib/bdii/gip/provider/cloud-info-provider</code> that calls the provider with the correct options for your site, for example:
<pre>#!/bin/sh
 
cloud-info-provider-service --yaml /etc/cloud-info-provider/openstack.yaml \
                            --middleware openstack \
                            --os-username &lt;username&gt; --os-password &lt;passwd&gt; \
                            --os-tenant-name &lt;tenant&gt; --os-auth-url &lt;url&gt;
</pre>
*Give execution permission:


===General cloud site information===
chmod +x /var/lib/bdii/gip/provider/cloud-info-provider
{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black;"
!Variable name
!Description
!Examples
|-
|provider['site_name']
|This is the name of the cloud site, if it was a Grid site it would have been the GOCDB name.
|Self explanatory
|-
|provider['production_level']
|State of the site certification (production level)
|production
|-
|provider['full_bdii_schema']
|Flag for production full BDII schema (set to true, for new BDII) or only cloud resources (set to false, for existing BDII)
|true
|-
|provider['www']
|Address of the information web site of the resource provider, or the cloud service. It should contain a full URL (eg. with http:// prefix)
|http://recas-pon.ba.infn.it/
|-
|provider['Country']
|Country where the cloud infrastructure is located (in [http://en.wikipedia.org/wiki/ISO_3166-1_alpha-2#Officially_assigned_code_elements ISO 3166-1 Alpha2 standard])
|IT
|-
|provider['site_longitude']
|The longitude of the main site location (in the XX.XXXX format)
|16.8891
|-
|provider['site_latitude']
|The latitude of the main site location
|41.1123
|-
|provider['affiliated_ngi']
|The name of the affiliated NGI
|NGI_IT
|-
|provider['user_support_contact']
|Site user support contact mail address
|admin@cloudsite.org
|-
|provider['general_contact']
|Site general contact mail address
|admin@cloudsite.org
|-
|provider['sysadmin_contact']
|Site system administration main contact mail address
|admin@cloudsite.org
|-
|provider['security_contact']
|Site security contact mail address
|admin@cloudsite.org
|-
|provider['sysadmin_contact']
|Site user support contact mail address
|admin@cloudsite.org
|-
|provider['site_bdii_host']
|Local site BDII host
|site-bdii.mysite.it
|-
|provider['site_bdii_port']
|Local site BDII port
|2170
|-
|provider['site_total_cpu_cores']
|The total number of CPU cores provided by the site
|300
|-
|provider['site_total_ram_gb']
|The total RAM provided by the site (in GB)
|600
|-
|provider['site_total_storage_gb']
|The total number of storage provided by the site (in GB)
|1024
|-
|provider['iaas_middleware']
|The name of the IaaS middleware
|OpenStack Nova
|-
|provider['iaas_middleware_version']
|The version of the IaaS middleware deployed
|havana
|-
|provider['iaas_middleware_developer']
|The developer of the IaaS middleware deployed
|OpenStack
|-
|provider['iaas_hypervisor']
|The hypervisor deployed in the IaaS middleware
|KVM
|-
|provider['iaas_hypervisor_version']
|The version of the hypervisor deployed in the IaaS middleware
|1.5.0
|-
|provider['iaas_capabilities']
|This variable contains a list of strings "('string1','string2','string3,..)", those strings describe the capabilities provided by the cloud IaaS service. Please note that the strings are not formalized anywhere, new labels should be agreed within the Task Force, for the moment.
|('cloud.managementSystem','cloud.vm.uploadImage')
|-
|provider['iaas_endpoints']
|List of the endpoints to reach the IaaS service. This is a list of python dictionaries. Basically the format is "({endpoint1},{endpoint2},{endpoint3}...). the structure of the single ''{endpoint}'' bits will be described in an additional table. If no IaaS service is provided, please leave this array empty.
|
|-
|provider['os_tpl']
|List of the OS templates available in the cloud IaaS service, that is the different virtual machines images available to be instantiated by the user. It is a list of python dictionaries. Basically the format is "({os_tpl1},{os_tpl2},{os_tpl3}...). the structure of the single ''{os_tpl}'' bits will be described in an additional table.
|
|-
|provider['resource_tpl']
|List of the Resource templates (flavors) available in the cloud IaaS service, that is the different virtual machines virtual hardware resources (RAM, CPU, etc...) available to be instantiated by the user. It is a list of python dictionaries. Basically the format is "({resource_tpl1},{resource_tpl2},{resource_tpl3}...). the structure of the single ''{resource_tpl}'' bits will be described in an additional table.
|
|-
|provider['staas_middleware']
|The name of the STaaS middleware
|OpenStack Swift
|-
|provider['staas_middleware_version']
|The version of the STaaS middleware deployed
|havana
|-
|provider['staas_middleware_developer']
|The developer of the STaaS middleware deployed
|OpenStack
|-
|provider['staas_capabilities']
|This variable contains a list of strings "('string1','string2','string3,..)", those strings describe the capabilities provided by the cloud STaaS service. Please note that the strings are not formalized anywhere, new labels should be agreed within the Task Force, for the moment.
|('cloud.data.upload')
|-
|provider['staas_endpoints']
|List of the endpoints to reach the STaaS service. This is a list of python dictionaries. Basically the format is "({endpoint1},{endpoint2},{endpoint3}...). the structure of the single ''{endpoint}'' bits will be described in an additional table. If no STaaS service is provided, please leave this array empty.
|}


*and test it:


===IaaS Endpoint instance===
/var/lib/bdii/gip/provider/cloud-info-provider
It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.


If a cloud IaaS service provide different interfaces, such as OCCI and EC2, it should publish different endpoints with different implementation.
*That should return the complete LDIF describing your site. Now you can start the bdii service


{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black;"
service bdii start
!Label
!Description
!Examples
|-
|endpoint_url
|The URL to reach the service endpoint, it should contain the protocol (e.g. ''https://'') and the port, if it is not the standard port for the protocol.
|https://one.cloud.gwdg.de:8443
|-
|endpoint_interface
|The interface implemented by the endpoint.
|OCCI
|-
|service_type_name/version/developer
|The scripts fills this with the 'iaas_middleware_name/version/developer' variables described above
|
|-
|interface_version
|The version of the specification of the interface implemented. The script fills this with interface['IaaS_api_version'] described above.
|interface['IaaS_api_version']
|-
|endpoint_technology
|This is the architecture implemented by the web service. The script fills this with interface['IaaS_api_endpoint_technology'] described above.
|interface['IaaS_api_endpoint_technology']
|-
|auth_method
|This is the AAI implemented by the web service. The script fills this with interface['IaaS_api_authorization_method'] described above.
|interface['IaaS_api_authorization_method']
|}


===OS_Tpl instance===
*And check that the information is being published
If a cloud service provide a list of default virtual images that can be instantiate by the user, they should be advertised with a list of execution environment.
It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.


{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black;"
ldapsearch -x -h localhost -p 2170 -b o=glue
!Label
!Description
!Examples
|-
|image_name
|name of the OS image. It should be self-explanatory and directly presentable to the user
|Scientific Linux 6.4 (x86_64)
|-
|image_version
|version of the OS image. Note that this is not the version of the OS, but the version of the virtual image
|1.0
|-
|marketplace_id
|URL of the image into the reference marketplace. For the Federated Cloud, this should point to the AppDB. To obtain this value from AppDB, go into the Virtual Machine image version page (eg. http://appdb.egi.eu/store/vm/image/2c24de6c-e385-49f1-b64f-f9ff35e70f43:9/ and copy the XML link provided by the interface)
|http://appdb.egi.eu/store/vm/image/2c24de6c-e385-49f1-b64f-f9ff35e70f43:9/xml
|-
|occi_id
|OCCI ID of the image on the site. It shall contain the full mixin to be used in the OCCI create compute API call.
|http://fedcloud.egi.eu/occi/infrastructure/os_tpl#ef13c0be-4de6-428f-ad5b-8f32b31a54a1
|-
|os_family
|This is the operating system type provided in the virtual image, the currently formalized families are: linux, windows, macosx, solaris
|linux
|-
|os_name
|The name of the operating system.
|suse
|-
|os_version
|The version of the operating system
|6.0.4
|-
|platform
|The processor architecture (i386,amd64,itanium, sparc, powerpc)
|amd64
|}


===Resource_Tpl instance===
== Add your resource BDII to the site-BDII  ==
If a cloud service provide a list of pre-defined virtual hardware resource templates (flavors) that can be instantiate by the user, they should be advertised with a list of resource templates.
It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.


{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black;"
Add your cloud-info-provider to your site-BDII by adding a new URL like this:  
!Label
!Description
!Examples
|-
|occi_id
|OCCI ID of the resource template on the site. It shall contain the full mixin to be used in the OCCI create compute API call.
|http://fedcloud.egi.eu/occi/infrastructure/resource_tpl#tiny-with-disk
|-
|memory
|The amount of RAM memory associated to the VM instance (MB)
|512
|-
|cpu
|The number of virtual CPU associated to the VM instance
|2
|-
|platform
|The virtual processor architecture (i386,amd64,itanium, sparc, powerpc)
|amd64
|-
|network
|The type of IP provided to the VM (public,private)
|public
|}


=== STaaS Endpoint instance===
ldap://&lt;cloud-info-provier-hostname&gt;:2170/GLUE2GroupID=cloud,o=glue
It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.


If a cloud IaaS service provide different interfaces, such as CDMI and S3, it should publish different endpoints with different implementation.
Check how to set up your Site-BDII at [[MAN01 How to publish Site Information]] for information on how to add the URL.  


{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black;"
[[Category:Operations_Manuals]]
!Label
!Description
!Examples
|-
|endpoint_url
|The URL to reach the service endpoint, it should contain the protocol (e.g. ''https://'') and the port, if it is not the standard port for the protocol.
|https://one.cloud.gwdg.de:8443
|-
|endpoint_interface
|The interface implemented by the endpoint.
|OCCI
|-
|service_type_name/version/developer
|The scripts fills this with the 'iaas_middleware_name/version/developer' variables described above
|
|-
|interface_version
|The version of the specification of the interface implemented. The script fills this with interface['staas_api_version'] described above.
|interface['staas_api_version']
|-
|endpoint_technology
|This is the architecture implemented by the web service. The script fills this with interface['STaaS_api_endpoint_technology'] described above.
|interface['STaaS_api_endpoint_technology']
|-
|auth_method
|This is the AAI implemented by the web service. The script fills this with interface['STaaS_api_authorization_method'] described above.
|interface['STaaS_api_authorization_method']
|}

Latest revision as of 13:36, 10 September 2021

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators
Alert.png This page is Deprecated; the content has been moved to https://docs.egi.eu/providers/cloud-compute/openstack/#egi-information-system


Purpose

This page provides instructions on how to configure the Federated Cloud Production resource-BDII.

Installation

Documentation and general installation guidelines for the cloud-info-provider are available at https://github.com/EGI-FCTF/cloud-bdii-provider

If you don't have already a BDII-site service in production, you will need to install also the bdii package (available in EPEL repo).

If you are using OpenStack as provider, you also need to install the python-novaclient package.

If you want to install from packages, here follow some guidelines.

RHEL/CentOS/ScientificLinux

wget http://repository.egi.eu/community/software/cloud.info.provider/0.x/releases/repofiles/sl-6-x86_64.repo \
   -O /etc/yum.repos.d/cloud-info-provider.repo
  • Install the package
yum install cloud-info-provider-service

For Debian/Ubuntu 6/7/8

  • Add AppDB's repository:
sudo wget http://repository.egi.eu/community/software/cloud.info.provider/0.x/releases/repofiles/ubuntu-precise-amd64.list \
        -O /etc/apt/sources.list.d/cloud-info-provider.list
  • Add AppDB key:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E2E992EB352D3E14
  • Install the package
sudo apt-get update
sudo apt-get install python-cloud-info-provider

Configuration

Use one of the template files in /etc/cloud-info-provider as basis for creating your own YAML file with the static information of your resources. E.g:

cp /etc/cloud-info-provider/sample.openstack.yaml /etc/cloud-info-provider/bdii.yaml

Each middleware has its own options to fetch the dynamic information, check the --help option for more information. Some additional notes are given in the following sections.

General

  • Site name will be fetched from site -> name in the template file. Set it to the name defined in GOCDB. Alternatively, the site name can be fetched from /etc/glite-info-static/site/site.cfg (or by the file set with the --glite-site-info-static option)

OpenNebula

  • Use the --middleware opennebularocci option to activate this provider
  • Use the sample.opennebularocci.yaml configuration sample. Change the following lines:
site:
    # Your site name, as in GODCB (if omitted or set to None, this value is
    # retreived from /etc/glite-info-static/site/site.cfg )
    #name: SITE_NAME
compute:
    # Total number of cores available
    total_cores: 0
    # Total RAM available (GB)
    total_ram: 0
    # Hypervisor name (e.g. KVM, Xen, etc.)
    hypervisor: Foo Hypervisor
    # Hypervisor version
    hypervisor_version: 0.0.0
    # ...
    # Middleware version
    middleware_version: 0.0
    endpoints:
        # ...

        # Host serving your Virtual Machine Management interface (rOCCI-server)
        https://cloud-service01.example.org:11443:
            endpoint_url: https://cloud-service01.example.org:11443
    # Images are retreived automatically by the endpoint
    images:
        defaults:
            #...
            # os_tpl schema prefix must match ROCCI_SERVER_OPENNEBULA_SCHEMA_NAMESPACE in rOCCI-server, only the host part
            schema: http://schemas.cloud-service01.example.org/occi/infrastructure/os_tpl
  • You need to specify connection parameters to the OpenNebula XML-RPC interface:
    • --on-auth parameter should contain the authorization parameters for an existing user with full read permissions on the image disks. If the user has been created with the core driver, this parameter shall be set to <username>:<password>.
    • --on-rpcxml-endpoint shall contain the address of the RPCv2 endpoint. Usually it is http://<hostname>:2633/RPC2. If not on a secure network, it is suggested to provide this interface via https, since the on-auth parameter will be sent in clear text to the server.
  • Compute templates can be gathered in two ways. One option does not preclude the other and the resulting templates will be the merge of the two.
    • directly from remote OpenNebula (with rOCCI-server installed and configured), by setting the --rocci-remote-templates flag
    • manually by placing them in the configuration file
  • With the --cloudkeeper-images flag, OS templates can be filtered so only the cloudkeeper ones are published.

OpenStack

  • Use the --middleware openstack option to activate this provider
  • The OpenStack provider uses python-novaclient (needs to be installed separately)
    • --os-username, --os-password, --auth-tenant-name, --os-auth-url, --os-cacert, --insecure options to the cloud-provider allow to set the connection parameters. Alternatively you can use environment variables (e.g. OS_USERNAME) as with other OpenStack clients
    • --insecure should not be used in production!
  • Be sure that keystone contains the OCCI endpoint, otherwise it will not be published by the BDII. You can check this via the command keystone service-list. To create a new service and endpoint, you can run
keystone service-create --name occi_api --type occi --description 'Nova OCCI Service'

keystone endpoint-create --service_id <service-id> --region RegionOne --publicurl http://$HOSTNAME:8787/ \
                                         --internalurl http://$HOSTNAME:8787/ --adminurl http://$HOSTNAME:8787/

where the service-id is the one obtained from keystone service-list.

  • By default, the provider script will filter images without marketplace uri defined into the marketplace or vmcatcher_event_ad_mpuri property. If you want to list all the images templates (included local snapshots), set the variable 'require_marketplace_id: false' under 'compute' -> 'images' -> 'defaults' in the YAML configuration file.

Create the provider

  • Create the file /var/lib/bdii/gip/provider/cloud-info-provider that calls the provider with the correct options for your site, for example:
#!/bin/sh

cloud-info-provider-service --yaml /etc/cloud-info-provider/openstack.yaml \
                            --middleware openstack \
                            --os-username <username> --os-password <passwd> \
                            --os-tenant-name <tenant> --os-auth-url <url>
  • Give execution permission:
chmod +x /var/lib/bdii/gip/provider/cloud-info-provider
  • and test it:
/var/lib/bdii/gip/provider/cloud-info-provider
  • That should return the complete LDIF describing your site. Now you can start the bdii service
service bdii start
  • And check that the information is being published
ldapsearch -x -h localhost -p 2170 -b o=glue

Add your resource BDII to the site-BDII

Add your cloud-info-provider to your site-BDII by adding a new URL like this:

ldap://<cloud-info-provier-hostname>:2170/GLUE2GroupID=cloud,o=glue

Check how to set up your Site-BDII at MAN01 How to publish Site Information for information on how to add the URL.