Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

HOWTO15 How to configure the Federated Cloud BDII

From EGIWiki
Jump to navigation Jump to search

Purpose

This page provides instructions on how to configure the Federated Cloud Production BDII.

Configuration guide

For sites with existing production BDII

If your site has already a production BDII (eg. for existing Grid resources), you need to add to the site your production cloud information. This information is in a separated dedicated three, so it will not interfere with your existing data. To add the production cloud information, you need to:

  1. Be sure that your LDAP supports GLUE2.0 schema and publish all the user relevant information on GLUE2.0 . Be sure also that the GOCDB GIIS entry of your production site points to the GLUE2.0 schema, (eg: ldap://prisma-cloud.ba.infn.it:2170/GLUE2DomainID=PRISMA-INFN-BARI,o=glue ). If that is not the case, update your BDII to GLUE2.0 and update the GIIS URL accordingly.
  2. Run the Python script to produce the cloud LDIF, setting the provider['full_bdii_schema'] variable to 'false' . More information on how to run the schema is in the following chapter
  3. Delete old cloud data (if any)(eg. $ ldapdelete -H ldap://full.hostname:2170 'GLUE2GroupID=cloud,GLUE2DomainID=SZTAKI,o=glue' -D 'your LDAP admin DN' -W )
  4. Add new cloud data (eg. ldapadd -f cloud-ldif.ldif -H ldap://full.hostname:2170 -D 'your LDAP admin DN' -W)

For sites without existing production BDII

If your site does not have a site BDII, you can setup a new one simply by setting up a new LDAP server. To do so:

  1. Install OpenLDAP packages (or any LDAP server you like) and configure it using the following configuration files:
  2. Follow the [#Generate_the_cloud_information Generate the cloud information] section to produce the cloud LDIF, setting the provider['full_bdii_schema'] variable to 'true' . More information on how to run the schema is in the following chapter
  3. Delete existing LDAP database directory, as specified in the slapd.conf file (if not already empty) (eg: rm -rf /var/db/openldap/openldap-data/) and create a new empty one (eg: mkdir /var/db/openldap/openldap-data/; chown ldap:ldap -R /var/db/openldap/openldap-data/)
  4. Load the ldap data (eg: $ slapadd -f /etc/openldap/slapd.conf -l cloud-ldif.ldif )
  5. Start the LDAP server (eg: $ slapd -f /etc/openldap/slapd.conf -h ldap://full.hostname:2170 -d 0 > file-name 2>&1 & or service slapd start)
  6. Configure your GOCDB site information the 'GIIS URL' with the address of your site BDII and the base schema, (eg: ldap://prisma-cloud.ba.infn.it:2170/GLUE2DomainID=PRISMA-INFN-BARI,o=glue )

Refresh the data

Data contained in the LDAP server is static, thus refresh or update need to be performed by hand. If you need to perform major updates, you can re-generate the entire cloud schema (using the Data publish script, as reported in the chapter below) and then re-import it. To do it, you can follow again the instruction in the [#configuration_guide Configuration Guide] above.

Generate the cloud information

Cloud information can be generated using the following Python script ((single-info-system-maker-extended-v3.2.py) Download here)

The script must be edited with your site and cloud services information before its execution, as described in the following tables. To run the script, just execute:

 $ chmod +x single-info-system-maker-extended-v3.2.py
 $ ./single-info-system-maker-extended-v3.2.py > cloud-ldif.ldif

General interface information

Variable name Description Examples
interface['IaaS_api'] API standard for the IaaS management interface OCCI
interface['IaaS_api_version'] API standard version for the IaaS management interface 1.1
interface['IaaS_api_endpoint_technology'] API endpoint technology for the IaaS management interface REST
interface['IaaS_api_authorization_method'] API AAI for the IaaS management interface X509-VOMS
interface['STaaS_api'] API standard for the STaaS management interface CDMI
interface['STaaS_api_version'] API standard version for the STaaS management interface 1.0.1
interface['STaaS_api_endpoint_technolog'] API endpoint technology for the STaaS management interface REST
interface['STaaS_api_authorization_method'] API AAI for the STaaS management interface X509-VOMS

General cloud site information

Variable name Description Examples
provider['site_name'] This is the name of the cloud site, if it was a Grid site it would have been the GOCDB name. Self explanatory
provider['production_level'] State of the site certification (production level) production
provider['full_bdii_schema'] Flag for production full BDII schema (set to true, for new BDII) or only cloud resources (set to false, for existing BDII) true
provider['www'] Address of the information web site of the resource provider, or the cloud service. It should contain a full URL (eg. with http:// prefix) http://recas-pon.ba.infn.it/
provider['Country'] Country where the cloud infrastructure is located (in ISO 3166-1 Alpha2 standard) IT
provider['site_longitude'] The longitude of the main site location (in the XX.XXXX format) 16.8891
provider['site_latitude'] The latitude of the main site location 41.1123
provider['affiliated_ngi'] The name of the affiliated NGI NGI_IT
provider['user_support_contact'] Site user support contact mail address admin@cloudsite.org
provider['general_contact'] Site general contact mail address admin@cloudsite.org
provider['sysadmin_contact'] Site system administration main contact mail address admin@cloudsite.org
provider['security_contact'] Site security contact mail address admin@cloudsite.org
provider['sysadmin_contact'] Site user support contact mail address admin@cloudsite.org
provider['site_bdii_host'] Local site BDII host site-bdii.mysite.it
provider['site_bdii_port'] Local site BDII port 2170
provider['site_total_cpu_cores'] The total number of CPU cores provided by the site 300
provider['site_total_ram_gb'] The total RAM provided by the site (in GB) 600
provider['site_total_storage_gb'] The total number of storage provided by the site (in GB) 1024
provider['iaas_middleware'] The name of the IaaS middleware OpenStack Nova
provider['iaas_middleware_version'] The version of the IaaS middleware deployed havana
provider['iaas_middleware_developer'] The developer of the IaaS middleware deployed OpenStack
provider['iaas_hypervisor'] The hypervisor deployed in the IaaS middleware KVM
provider['iaas_hypervisor_version'] The version of the hypervisor deployed in the IaaS middleware 1.5.0
provider['iaas_capabilities'] This variable contains a list of strings "('string1','string2','string3,..)", those strings describe the capabilities provided by the cloud IaaS service. Please note that the strings are not formalized anywhere, new labels should be agreed within the Task Force, for the moment. ('cloud.managementSystem','cloud.vm.uploadImage')
provider['iaas_endpoints'] List of the endpoints to reach the IaaS service. This is a list of python dictionaries. Basically the format is "({endpoint1},{endpoint2},{endpoint3}...). the structure of the single {endpoint} bits will be described in an additional table. If no IaaS service is provided, please leave this array empty.
provider['os_tpl'] List of the OS templates available in the cloud IaaS service, that is the different virtual machines images available to be instantiated by the user. It is a list of python dictionaries. Basically the format is "({os_tpl1},{os_tpl2},{os_tpl3}...). the structure of the single {os_tpl} bits will be described in an additional table.
provider['resource_tpl'] List of the Resource templates (flavors) available in the cloud IaaS service, that is the different virtual machines virtual hardware resources (RAM, CPU, etc...) available to be instantiated by the user. It is a list of python dictionaries. Basically the format is "({resource_tpl1},{resource_tpl2},{resource_tpl3}...). the structure of the single {resource_tpl} bits will be described in an additional table.
provider['staas_middleware'] The name of the STaaS middleware OpenStack Swift
provider['staas_middleware_version'] The version of the STaaS middleware deployed havana
provider['staas_middleware_developer'] The developer of the STaaS middleware deployed OpenStack
provider['staas_capabilities'] This variable contains a list of strings "('string1','string2','string3,..)", those strings describe the capabilities provided by the cloud STaaS service. Please note that the strings are not formalized anywhere, new labels should be agreed within the Task Force, for the moment. ('cloud.data.upload')
provider['staas_endpoints'] List of the endpoints to reach the STaaS service. This is a list of python dictionaries. Basically the format is "({endpoint1},{endpoint2},{endpoint3}...). the structure of the single {endpoint} bits will be described in an additional table. If no STaaS service is provided, please leave this array empty.


IaaS Endpoint instance

It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.

If a cloud IaaS service provide different interfaces, such as OCCI and EC2, it should publish different endpoints with different implementation.

Label Description Examples
endpoint_url The URL to reach the service endpoint, it should contain the protocol (e.g. https://) and the port, if it is not the standard port for the protocol. https://one.cloud.gwdg.de:8443
endpoint_interface The interface implemented by the endpoint. OCCI
service_type_name/version/developer The scripts fills this with the 'iaas_middleware_name/version/developer' variables described above
interface_version The version of the specification of the interface implemented. The script fills this with interface['IaaS_api_version'] described above. interface['IaaS_api_version']
endpoint_technology This is the architecture implemented by the web service. The script fills this with interface['IaaS_api_endpoint_technology'] described above. interface['IaaS_api_endpoint_technology']
auth_method This is the AAI implemented by the web service. The script fills this with interface['IaaS_api_authorization_method'] described above. interface['IaaS_api_authorization_method']

OS_Tpl instance

If a cloud service provide a list of default virtual images that can be instantiate by the user, they should be advertised with a list of execution environment. It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.

Label Description Examples
image_name name of the OS image. It should be self-explanatory and directly presentable to the user Scientific Linux 6.4 (x86_64)
image_version version of the OS image. Note that this is not the version of the OS, but the version of the virtual image 1.0
marketplace_id URL of the image into the reference marketplace. For the Federated Cloud, this should point to the AppDB. To obtain this value from AppDB, go into the Virtual Machine image version page (eg. http://appdb.egi.eu/store/vm/image/2c24de6c-e385-49f1-b64f-f9ff35e70f43:9/ and copy the XML link provided by the interface) http://appdb.egi.eu/store/vm/image/2c24de6c-e385-49f1-b64f-f9ff35e70f43:9/xml
occi_id OCCI ID of the image on the site. It shall contain the full mixin to be used in the OCCI create compute API call. http://fedcloud.egi.eu/occi/infrastructure/os_tpl#ef13c0be-4de6-428f-ad5b-8f32b31a54a1
os_family This is the operating system type provided in the virtual image, the currently formalized families are: linux, windows, macosx, solaris linux
os_name The name of the operating system. suse
os_version The version of the operating system 6.0.4
platform The processor architecture (i386,amd64,itanium, sparc, powerpc) amd64

Resource_Tpl instance

If a cloud service provide a list of pre-defined virtual hardware resource templates (flavors) that can be instantiate by the user, they should be advertised with a list of resource templates. It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.

Label Description Examples
occi_id OCCI ID of the resource template on the site. It shall contain the full mixin to be used in the OCCI create compute API call. http://fedcloud.egi.eu/occi/infrastructure/resource_tpl#tiny-with-disk
memory The amount of RAM memory associated to the VM instance (MB) 512
cpu The number of virtual CPU associated to the VM instance 2
platform The virtual processor architecture (i386,amd64,itanium, sparc, powerpc) amd64
network The type of IP provided to the VM (public,private) public

STaaS Endpoint instance

It is a python dictionary, the format is: {'label1':'value1' , 'label2':'value2'....}.

If a cloud IaaS service provide different interfaces, such as CDMI and S3, it should publish different endpoints with different implementation.

Label Description Examples
endpoint_url The URL to reach the service endpoint, it should contain the protocol (e.g. https://) and the port, if it is not the standard port for the protocol. https://one.cloud.gwdg.de:8443
endpoint_interface The interface implemented by the endpoint. OCCI
service_type_name/version/developer The scripts fills this with the 'iaas_middleware_name/version/developer' variables described above
interface_version The version of the specification of the interface implemented. The script fills this with interface['staas_api_version'] described above. interface['staas_api_version']
endpoint_technology This is the architecture implemented by the web service. The script fills this with interface['STaaS_api_endpoint_technology'] described above. interface['STaaS_api_endpoint_technology']
auth_method This is the AAI implemented by the web service. The script fills this with interface['STaaS_api_authorization_method'] described above. interface['STaaS_api_authorization_method']