Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "MAN10"

From EGIWiki
Jump to navigation Jump to search
Line 84: Line 84:
* If providing CDMI access (Object storage):
* If providing CDMI access (Object storage):
** Swift
** Swift
###''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">'''Required by EGI configuration options for openstack''': if there is anything required by EGI it should be written here: e.g. </span></span>''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">list of OpenStack components required by EGI, how to implement EGI security requirements on OpenStack, ''suggested network schema for Openstack - GRE or VXLAN.</span></span><br>''OpenStack installation you need to do by yourself following instructions here:&nbsp;[http://docs.openstack.org/ http://docs.openstack.org/] <span>&nbsp;</span>


==== AAI integration in OpenStack ====
==== AAI integration in OpenStack ====

Revision as of 10:42, 9 March 2015

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home • Manuals • Procedures • Training • Other • Contact ► For: VO managers • Administrators



Title Setting up Cloud Resource Centre
Document link https://wiki.egi.eu/wiki/MAN10
Last modified 19 August 2014
Policy Group Acronym OMB
Policy Group Name Operations Management Board
Contact Group operations-support@mailman.egi.eu
Document Status DRAFT
Approved Date
Procedure Statement This manual provides information on how to set up Cloud Resource Centre.
Owner Owner of procedure



Introduction

EGI cloud supports 3 middlewares. It means you can base your cloud site installation on one of the following cloud software:

  • OpenNebula
  • OpenStack
  • Synnefo

If you want to install an EGI Cloud Site please have a look at our EGI Cloud Site Installation Manuals below.

Note: EGI Cloud Site Installation Manual is a step-by-step instruction for Cloud Site Admin. The manual is not meant to be a comprehensive on topics related to the installation, it is a collection of steps taken by someone to install an EGI cloud site starting from a scratch. Commands executed should be made available for someone to copy&paste and easily follow up. At some initial stage the manual may not cover all cases, but it is meant to be extended by other site admins while following up the manual. It is a living document.

The manuals

Current issues:

  • Documentation for cloud components is written with assumption that the admin knows where (machine, neighbour components) this components should be installed. It is missing the general cloud site deployment context
  • Documentation should address the prerequsities part. 
  • Documentation should address the constraints and limitations part i.e. supported
  • Documentation should provide a contact person (per component) which can be contacted in case of questions/problems.  
  • Documentation should provide commands for checking validity of installation.


Prerequisities & Limitations

Whatever cloud stack you choose you need to prepare some things at the begining:

  1. Hardware (minimal hw requirements for small cloud site e.g up to 100 VMs):
    1. number of physical machines, performance/capacity requirements: RAM size
    2. disk space - how big, where must be connected, performance of network links (images are heavy!)
  2. DNS names, X.509 certificates
  3. Register in fedcloud VO
  4. Registration in AppDB to have access to private EGI VM image repository
  5. What operating systems are supported

Cloud management frameworks

OpenStack

Baustelle.png This part is under construction.


EGI Cloud site can be based on OpenStack software with some EGI extensions. See deployment schema (Note: high level description on what modules are to be put on which machines.)

OpenStack installation

Integration with FedCloud requires a working OpenStack installation. Follow the general documentation at http://docs.openstack.org/, there are packages ready to use for most distributions (check for example RDO for RedHat based distributions).

Requirements and Limitations

OpenStack integration with FedCloud is known to work with the following versions of OpenStack:

  • Havana (EOL by OpenStack, should not be used in production)
  • Icehouse
  • Juno

Suggested list of services to provide FedCloud integration:

  • Keystone service must be available in any case.
  • If providing OCCI access (VM management):
  • If providing CDMI access (Object storage):
    • Swift

AAI integration in OpenStack

Every FedCloud site must support authentication of users with X.509 certificates with VOMS extensions. The Keystone-VOMS extension enables this kind of authentication on Keystone.

  • Pre-requisites: you will need a valid host certificate from a EUGridPMA CA.
  • Installation: documentation on the installation is available at Keystone-voms documentation. Make sure to use the correct documentation for your OpenStack version.
  • Take into account that using keystone-voms plugin will enforce the use of https for your Keystone service, you will need to update your URLs at the Keystone catalog and in the configuration of your services (check [keystone_authtoken] in nova, cinder, glance config files and any other service that needs to check keystone tokens)

change database? other services

  • VOs: Every FedCloud site is expected to support fedcloud.egi.eu, dteam and ops VOs. You should configure this VO in your /etc/keystone/voms.json file. Make sure that the tenant you are mapping the VO to exists. Below there is a sample voms.json file, adapt it with the appropriate names of your tenants:
{
    "fedcloud.egi.eu": {
        "tenant": "VO:fedcloud.egi.eu"
    },
    "dteam": {
        "tenant": "VO:dteam"
    },
    "ops": {
        "tenant": "VO:ops"
    }
}

You also need to include the appropriate .lsc files for each VO at /etc/grid-security/vomsdir/: 

mkdir -p /etc/grid-security/vomsdir/fedcloud.egi.eu

cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms1.egee.cesnet.cz << EOF
/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms1.egee.cesnet.cz
/C=NL/O=TERENA/CN=TERENA eScience SSL CA
EOF

cat > /etc/grid-security/vomsdir/fedcloud.egi.eu/voms2.grid.cesnet.cz << EOF
/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms2.grid.cesnet.cz
/C=NL/O=TERENA/CN=TERENA eScience SSL CA
EOF

mkdir -p /etc/grid-security/vomsdir/dteam

cat > /etc/grid-security/vomsdir/dteam/voms.hellasgrid.gr << EOF
/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2006
EOF

cat > /etc/grid-security/vomsdir/dteam/voms2.hellasgrid.gr << EOF
/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2006
EOF

mkdir -p /etc/grid-security/vomsdir/ops

cat > /etc/grid-security/vomsdir/dteam/lcg-voms2.cern.ch << EOF
/DC=ch/DC=cern/OU=computers/CN=lcg-voms2.cern.ch
/DC=ch/DC=cern/CN=CERN Grid Certification Authority
EOF

cat > /etc/grid-security/vomsdir/dteam/voms2.cern.ch << EOF
/DC=ch/DC=cern/OU=computers/CN=voms2.cern.ch
/DC=ch/DC=cern/CN=CERN Grid Certification Authority
EOF

OCCI Support

OCCI is the EGI-approved access method for computing resources that every cloud site must expose. OCCI-OS is the recommended software to provide this capability.

    1. OCCI installation is described here: https://github.com/EGI-FCTF/occi-os/ but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and missing commands to be executed.
    1. Another nice installation guide (which one we should follow) is here  https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d do we want to provide people with two instructions?

EGI Image Management

  1. EGI Image Management
    Each cloud site must give access to EGI-approved VM images. An image clarifying functions and relations between vmcaster, vmcatcher, glance, glancepush and openstack handler for vmcatcher would be well welcome.
    1. Registration in AppDB to have access to private EGI VM image repository - missing, please describe steps to be done by Site Admin, should be as prerequisite steps because it requires manual step by someone
    2. VMCatcher - allows users to subscribe to VMs (unclear). https://github.com/hepix-virtualisation/vmcatcher
    3. Install EGI-customized version of glancepush: instruction: https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack software repo: https://appdb.egi.eu/store/software/python.glancepush/releases/0.0.x
    4. Install Openstack handler for vmcatcher: instruction: : https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack software repo: https://appdb.egi.eu/store/software/openstack.handler.for.vmcatcher

EGI Accounting

  1. EGI Accounting
    Each cloud site must publish utilization data  EGI acccounting database which is APEL repository
    1. Instruction: https://github.com/EGI-FCTF/osssm/wiki

EGI Information System

  1. EGI Information System
    Each cloud site must publish information to EGI information system which is based on BDII
    1. Instruction: https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions#OpenStack- site admin doing OpenStack do not want to read how it works for OpenNebula

Registration of services in GOCDB

  1. EGI Configuration Management Database (GOCDB)
    Each cloud site must register services in EGI configuration management database which is GOCDB
    Need information if cloud site must be separate from grid site or can be share. Shouldn't GOCDB step be earlier to allow nagios monitoring?
    1. Registering endpoints https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario5#GOCDB - ok, but we need info on what exactly has been registered in gocDB like "I have regiesterd os.acme.org as with type eu.egi.cloud.accounting.
    2. Registering SiteExtension Properties - is this still valid requirement? missing instruction

Installation Validation

  1. Installation validaton - this is a new step: describe steps performed by site admin that confirms the site installation is working well according to EGI requirements. It is better to have it as a separate, final step for all checks
    1. Nagios step - missing
    2. check accounting - missing
    3. check vmcatcher subscription - missing
    4. check BDII publishing - missing
    5. check OCCI It is possible to reuse https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Compute_.28OCCI.29_checks
    6. check CDMI It is possible to reuse https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Storage_.28CDMI.29_checks

Support for CDMI Maybe it is better to have separated part on CDMI and not mix it with OCCI.

OpenNebula

EGI Cloud site is based on OpenNebula software with some EGI extensions.See Deployment Schema (Note: here we need high level explanation on what modules are to be put on which machines.)

Stages of installation (similar for every middleware):

  1. OpenNebula installation with X.509 support
    1. Be consistent on requirements WHICH OpenNebula version is supported.
    2. Unfortunately this manual does not cover OpenNebula installation. You need to do this by yourself but this is well described here: http://docs.opennebula.org/4.4/
    3. Configure X.509 support according to http://docs.opennebula.org/4.4/administration/authentication/x509_auth.html
  2. Support for OCCI - EGI-approved access method
    1. Described here: https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and which commands executed.
  3. EGI User Authentication/Authorization
    1. You need to integrate with Perun. Described here https://github.com/EGI-FCTF/fctf-perun but missing context of EGI Cloud site installation and missing commands to be executed.
  4. EGI Image Management
    1. Missing in the instructions for OpenNebula. Seems we have in https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#VMcatcher section for Opennebula. Still no common context.
  5. EGI Accounting
    1. Described here: https://github.com/EGI-FCTF/opennebula-cloudacc but missing context of EGI Cloud site installation - complete with info on which host the commands should be executed.
  6. EGI Information System
    1. Described here https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions but again missing context of where these commands should be executed.
  7. EGI Configuration Management Database (GOCDB)
    1. Manual not available. We need information on OpenNebula-specific service types to be registered in GOCDB.

What with support for CDMI in Opennebula?

Synnefo

There are installation guides e.g. https://www.synnefo.org/docs/synnefo/latest/install-guide-debian.html#install-guide-debian but it seems there is no EGI-specific installation guides.

Revision History

Version Authors Date Comments
Retrieved from "https://wiki.egi.eu/w/index.php?title=MAN10&oldid=77638"