Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "MAN10"

From EGIWiki
Jump to navigation Jump to search
Line 13: Line 13:
}}  
}}  


==
= Introduction =


EGI cloud supports 3 middlewares. It means you can base your cloud site installation on one of the following cloud software:


*OpenNebula
= Introduction  =
*OpenStack
 
EGI cloud supports 3 middlewares. It means you can base your cloud site installation on one of the following cloud software:
 
*OpenNebula  
*OpenStack  
*Synnefo
*Synnefo


If you want to install an EGI Cloud Site please have a look at our EGI Cloud Site Installation Manuals below.
If you want to install an EGI Cloud Site please have a look at our EGI Cloud Site Installation Manuals below.  
 
''<span style="color: rgb(51,102,255);">Note:</span>''
 
EGI Cloud Site Installation Manual is a step-by-step instruction for
 
Cloud Site Admin. The manual is not meant to be a comprehensive on a topics related to the installation, it is a collection of steps taken by someone to install an EGI cloud site starting from a scratch. Commands
 
executed should be made available for someone to copy&amp;paste and easily follow up. At some initial stage the manual may not cover all cases, but it is meant to be extended by other site admins while following up the manual. It is a living document.<br>
 
= The manuals  =


''<span style="color: rgb(51,102,255);">Note:
<span style="color: rgb(51,102,255);">'''''Current issues:'''''</span>  
EGI Cloud Site Installation Manual is a step-by-step instruction for
Cloud Site Admin. The manual is not meant to be a comprehensive on a
topics related to the installation, it is a collection of steps taken by
someone to install an EGI cloud site starting from a scratch. Commands
executed should be made available for someone to copy&amp;paste and
easily follow up. At some initial stage the manual may not cover all
cases, but it is meant to be extended by other site admins while
following up the manual. It is a living document.</span><br>''


= The manuals =
*<span style="color: rgb(51,102,255);">''Documentation''
</span> for cloud components is written with assumption that the admin knows where (machine, neighbour components) this components should be installed. It is missing the general cloud site deployment context which


<span style="color: rgb(51,102,255);">'''''Current issues:'''''</span>


*<span style="color: rgb(51,102,255);">''Documentation
*<span style="color: rgb(51,102,255);">''Documentation should address the prerequsities part.''</span><span style="color: rgb(51,102,255);">&nbsp;</span>  
for cloud components is written with assumption that the admin knows
*<span style="color: rgb(51,102,255);">''Documentation''
where (machine, neighbour components) this components should be
</span> should address the constraints and limitations part i.e. supported  
installed. It is missing the general cloud site deployment context which
a manual should keep.''</span>
*<span style="color: rgb(51,102,255);">''Documentation should address the prerequsities part.''</span><span style="color: rgb(51,102,255);">&nbsp;</span>
*<span style="color: rgb(51,102,255);">''Documentation
should address the constraints and limitations part i.e. supported  
operating systems, supported software versions.&nbsp;&nbsp;''</span>
*<span style="color: rgb(51,102,255);">''Documentation should provide a contact person (per component) which can be contacted in case of questions/problems.&nbsp;''</span><span style="color: rgb(51,102,255);">&nbsp;</span>
*<span style="color: rgb(51,102,255);">''Documentation should provide commands for checking validity of installation.''</span><span style="color: rgb(51,102,255);">''
''</span>


== Prerequisities &amp; Limitations ==


Whatever cloud stack you choose you need to prepare some things at the begining:
*<span style="color: rgb(51,102,255);">''Documentation should provide a contact person (per component) which can be contacted in case of questions/problems.&nbsp;''</span><span style="color: rgb(51,102,255);">&nbsp;</span>
*<span style="color: rgb(51,102,255);">''Documentation should provide commands for checking validity of installation.''</span><span style="color: rgb(51,102,255);">
</span>


#Hardware (minimal hw requirements for small cloud site e.g up to 100 VMs): <br>
 
##number of physical machines, performance/capacity requirements: RAM size
== Prerequisities &amp; Limitations  ==
##disk space - how big, where must be connected, performance of network links (images are heavy!)
 
#DNS names, X.509 certificates
Whatever cloud stack you choose you need to prepare some things at the begining:
#Register in fedcloud VO
 
#Registration in AppDB to have access to private EGI VM image repository
#Hardware (minimal hw requirements for small cloud site e.g up to 100 VMs): <br>  
##number of physical machines, performance/capacity requirements: RAM size  
##disk space - how big, where must be connected, performance of network links (images are heavy!)  
#DNS names, X.509 certificates  
#Register in fedcloud VO  
#Registration in AppDB to have access to private EGI VM image repository  
#What operating systems are supported
#What operating systems are supported


= OpenStack =
= OpenStack =
 
EGI Cloud site can be based on OpenStack software with some EGI extensions. See deployment schema (''Note: <span style="color: rgb(51,102,255);">high level description on what modules are to be put on which machines.</span>'')


EGI Cloud site can be based on OpenStack software with some EGI extensions. See deployment schema (''Note: <span style="color: rgb(51,102,255);">high level description on what modules are to be put on which machines.</span>'')
Installation steps'':''  


Installation steps'':''
#OpenStack installation<br>
##'''''Requirements''''''/lim''''''itations:'''''
###''<span style="color: rgb(51,102,255);">'''Versions:''' Info on which OpenStack versions are supported by EGI on which operating system.&nbsp;<span style="color: rgb(51,102,255);">&nbsp;</span></span>''''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">&nbsp;</span></span>'''''
###''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">'''Required by EGI configuration options for openstack''': if there is anything required by EGI it should be written here: e.g. </span></span>''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">list of OpenStack components required by EGI, how to implement EGI security requirements on OpenStack, </span></span>''suggested network schema for Openstack - GRE or VXLAN.<br>''OpenStack installation you need to do by yourself following instructions here:&nbsp;[http://docs.openstack.org/ http://docs.openstack.org/] <span>&nbsp;</span>
#Support for OCCI in OpenStack<br>OCCI is the EGI-approved access method for computing resources that every cloud sites must expose.<br>
##<span style="color: rgb(51,102,255);">''OCCI installation is described here:&nbsp;[https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula <span style="color: rgb(51,102,255);">https://github.com/EGI-FCTF/occi-os/</span>] but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and missing commands to be executed.''</span>
##<span style="color: rgb(51,102,255);">''Another nice installation guide (which one we should follow) is here&nbsp; [https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d6 <span style="color: rgb(51,102,255);">https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d</span>][https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d6 <span style="color: rgb(51,102,255);">6</span>] do we want to provide people with two instructions?''</span>
#EGI User Authentication/Authorization<br>Each cloud site must support users with X.509 certificates from fedcloud VO<br>
##You need to have keystone-voms configured. [http://ifca.github.io/keystone-voms/ http://ifca.github.io/keystone-voms/]<span style="color: rgb(51,102,255);">''Missing context, missing command lines. Says it works for Grizzly, what with Havana, Icehouse?''</span>''<br>''
#EGI Image Management<br>Each cloud site must give access to EGI-approved VM images. ''<span style="color: rgb(51,102,255);">An</span>'' image clarifying functions and relations between vmcaster, vmcatcher, glance, glancepush and openstack handler for vmcatcher would be well welcome.''<br>''  


#OpenStack installation<br>
##Registration in AppDB to have access to private EGI VM image repository - ''<span style="color: rgb(51,102,255);">missin</span><span style="color: rgb(51,102,255);">g, please describe steps to be done by Site Admin, should be as prerequisite steps because it requires manual step by someone</span>''  
##'''''Requirements''''''/lim''''''itations:'''''
##<s>''VMCatcher - allows users to subscribe to VMs (unclear). [https://github.com/hepix-virtualisation/vmcatcher https://github.com/hepix-virtualisation/vmcatcher]''</s>  
###''<span style="color: rgb(51,102,255);">'''Versions:''' Info on which OpenStack versions are supported by EGI on which operating system.&nbsp;<span style="color: rgb(51,102,255);">&nbsp;</span></span>''''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">&nbsp;</span></span>''
##Install EGI-customized version of glancepush: instruction: [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack] software repo: [https://appdb.egi.eu/store/software/python.glancepush/releases/0.0.x https://appdb.egi.eu/store/software/python.glancepush/releases/0.0.x]  
###''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">'''Required by EGI configuration options for openstack''': if there is anything required by EGI it should be written here: e.g. ''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">list of OpenStack components required by EGI, how to implement EGI security requirements on OpenStack, </span></span>''suggested network schema for Openstack - GRE or VXLAN.</span></span><br>''OpenStack installation you need to do by yourself following instructions here:&nbsp;[http://docs.openstack.org/ http://docs.openstack.org/] <span>&nbsp;</span>
##Install <span class="external text">Openstack handler for vmcatcher</span>: instruction:&nbsp;: [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack] software repo: [https://appdb.egi.eu/store/software/openstack.handler.for.vmcatcher https://appdb.egi.eu/store/software/openstack.handler.for.vmcatcher]  
#Support for OCCI in OpenStack<br>OCCI is the EGI-approved access method for computing resources that every cloud sites must expose.<br>
#EGI Accounting<br>Each cloud site must publish utilization data&nbsp; EGI acccounting database which is APEL repository  
##<span style="color: rgb(51,102,255);">''OCCI installation is described here:&nbsp;[https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula <span style="color: rgb(51,102,255);">https://github.com/EGI-FCTF/occi-os/</span>] but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and missing commands to be executed.''</span>
##Instruction: [https://github.com/EGI-FCTF/osssm/wiki https://github.com/EGI-FCTF/osssm/wiki]  
##<span style="color: rgb(51,102,255);">''Another nice installation guide (which one we should follow) is here&nbsp; [https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d6 <span style="color: rgb(51,102,255);">https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d</span>][https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d6 <span style="color: rgb(51,102,255);">6</span>] do we want to provide people with two instructions?''</span>
#EGI Information System<br>Each cloud site must publish information to EGI information system which is based on BDII  
#EGI User Authentication/Authorization<br>Each cloud site must support users with X.509 certificates from fedcloud VO<br>
##Instruction: [https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions#OpenStack]''- <span style="color: rgb(51,102,255);">site admin doing OpenStack do not want to read how it works for OpenNebula</span>''  
##You need to have keystone-voms configured. [http://ifca.github.io/keystone-voms/ http://ifca.github.io/keystone-voms/]<span style="color: rgb(51,102,255);">''Missing context, missing command lines. Says it works for Grizzly, what with Havana, Icehouse?''</span>''<br>''
#EGI Configuration Management Database (GOCDB)<br>Each cloud site must register services in EGI configuration management database which is GOCDB<br>''<span style="color: rgb(51,102,255);">Need information if cloud site must be separate from grid site or can be share.</span>''<span style="color: rgb(51,102,255);">'' Shouldn't GOCDB step be earlier to allow nagios monitoring?''</span><br>  
#EGI Image Management<br>Each cloud site must give access to EGI-approved VM images. ''<span style="color: rgb(51,102,255);">An
##''[https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario5#GOCDB Registering endpoints https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario5#GOCDB] - <span style="color: rgb(51,102,255);">ok, but we need info on what exactly has been registered in gocDB like "I have regiesterd os.acme.org as</span>''<span style="color: rgb(51,102,255);"> with type eu.egi.cloud.accounting.</span>  
image clarifying functions and relations between vmcaster, vmcatcher,
##Registering SiteExtension Properties -''<span style="color: rgb(51,102,255);"> is this still valid requirement? missing instruction</span>''  
glance, glancepush and openstack handler for vmcatcher would be well
#<span style="color: rgb(51,102,255);">Installation validaton - this is a new step: ''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);"><span style="color: rgb(51,102,255);">describe steps performed by site admin that confirms the site installation is working well according to EGI requirements</span></span></span>''<span style="color: rgb(51,102,255);">.&nbsp;''It''
welcome.</span>''<br>
</span></span> is better to have it as a separate, final step for all checks since a)  
##Registration in AppDB to have access to private EGI VM image repository - ''<span style="color: rgb(51,102,255);">missin</span><span style="color: rgb(51,102,255);">g, please describe steps to be done by Site Admin, should be as prerequisite steps because it requires manual step by someone</span>''
 
##<s>''VMCatcher - allows users to subscribe to VMs (unclear). [https://github.com/hepix-virtualisation/vmcatcher https://github.com/hepix-virtualisation/vmcatcher]''</s>
##<span style="color: rgb(51,51,51);">Nagios step - ''<span style="color: rgb(51,102,255);">missing</span>''</span>  
##Install EGI-customized version of glancepush: instruction: [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack] software repo: [https://appdb.egi.eu/store/software/python.glancepush/releases/0.0.x https://appdb.egi.eu/store/software/python.glancepush/releases/0.0.x]
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check accounting <span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">''<span style="color: rgb(51,102,255);">- missing</span>''</span></span><br></span></span>  
##Install <span class="external text">Openstack handler for vmcatcher</span>: instruction: : [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack] software repo: [https://appdb.egi.eu/store/software/openstack.handler.for.vmcatcher https://appdb.egi.eu/store/software/openstack.handler.for.vmcatcher]
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check vmcatcher subscription <span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">''<span style="color: rgb(51,102,255);">- missing</span>''</span></span><br></span></span>  
#EGI Accounting<br>Each cloud site must publish utilization data&nbsp; EGI acccounting database which is APEL repository
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check BDII publishing ''<span style="color: rgb(51,102,255);">- missing</span>''<br></span></span>  
##Instruction: [https://github.com/EGI-FCTF/osssm/wiki https://github.com/EGI-FCTF/osssm/wiki]
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check OCCI ''[https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Compute_.28OCCI.29_checks It is possible to reuse https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Compute_.28OCCI.29_checks]''</span></span>  
#EGI Information System<br>Each cloud site must publish information to EGI information system which is based on BDII
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check CDMI <span style="color: rgb(51,102,255);">''It is possible to reuse [https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Storage_.28CDMI.29_checks <span style="color: rgb(51,102,255);">https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Storage_.28CDMI.29_checks</span>]''</span></span></span>  
##Instruction: [https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions#OpenStack]''- <span style="color: rgb(51,102,255);">site admin doing OpenStack do not want to read how it works for OpenNebula</span>''
#EGI Configuration Management Database (GOCDB)<br>Each cloud site must register services in EGI configuration management database which is GOCDB<br>''<span style="color: rgb(51,102,255);">Need information if cloud site must be separate from grid site or can be share.</span>''<span style="color: rgb(51,102,255);">'' Shouldn't GOCDB step be earlier to allow nagios monitoring?''</span><br>
##''[https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario5#GOCDB Registering endpoints https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario5#GOCDB] - <span style="color: rgb(51,102,255);">ok, but we need info on what exactly has been registered in gocDB like "I have regiesterd os.acme.org as</span>''<span style="color: rgb(51,102,255);"> with type eu.egi.cloud.accounting.</span>
##Registering SiteExtension Properties -''<span style="color: rgb(51,102,255);"> is this still valid requirement? missing instruction</span>''
#<span style="color: rgb(51,102,255);">Installation validaton - this is a new step: ''<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);"><span style="color: rgb(51,102,255);">describe steps performed by site admin that confirms the site installation is working well according to EGI requirements</span></span></span>''<span style="color: rgb(51,102,255);">.&nbsp;''It
is better to have it as a separate, final step for all checks since a)  
some checks cannot be performed immediately after configuration b) some
installation steps may override (affect negatively) results of previous
steps. ''</span></span>
##<span style="color: rgb(51,51,51);">Nagios step - ''<span style="color: rgb(51,102,255);">missing</span>''</span>
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check accounting <span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">''<span style="color: rgb(51,102,255);">- missing</span>''</span></span><br></span></span>
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check vmcatcher subscription <span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">''<span style="color: rgb(51,102,255);">- missing</span>''</span></span><br></span></span>
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check BDII publishing ''<span style="color: rgb(51,102,255);">- missing</span>''<br></span></span>
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check OCCI ''[https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Compute_.28OCCI.29_checks It is possible to reuse https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Compute_.28OCCI.29_checks]''</span></span>
##<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);">check CDMI <span style="color: rgb(51,102,255);">''It is possible to reuse [https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Storage_.28CDMI.29_checks <span style="color: rgb(51,102,255);">https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Storage_.28CDMI.29_checks</span>]''</span></span></span>
#<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);"><span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">Support for CDMI  
#<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);"><span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">Support for CDMI  
</span></span></span></span><span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);"><span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">''I'd suggest to have separated part on CDMI and not mix it with OCCI. ''
</span></span></span></span>
</span></span></span></span>


== OpenNebula ==
<span style="color: rgb(51,102,255);"><span style="color: rgb(51,51,51);"><span style="color: rgb(51,102,255);"><span style="color: rgb(51,102,255);">''I'd suggest to have separated part on CDMI and not mix it with OCCI. ''
</span></span></span></span>


EGI Cloud site is based on OpenNebula software with some EGI extensions.See Deployment Schema (''Note: here we need high level explanation on what modules are to be put on which machines.'')
== OpenNebula ==


Stages of installation ''(similar for every middleware):''
EGI Cloud site is based on OpenNebula software with some EGI extensions.See Deployment Schema (''Note: here we need high level explanation on what modules are to be put on which machines.'')


#OpenNebula installation with X.509 support
Stages of installation ''(similar for every middleware):''
##''Be consistent on requirements WHICH OpenNebula version is supported.''
 
##Unfortunately this manual does not cover OpenNebula installation. You need to do this by yourself but this is well described here: [http://docs.opennebula.org/4.4/ http://docs.opennebula.org/4.4/]
#OpenNebula installation with X.509 support  
##Configure X.509 support according to [http://docs.opennebula.org/4.4/administration/authentication/x509_auth.html http://docs.opennebula.org/4.4/administration/authentication/x509_auth.html]
##''Be consistent on requirements WHICH OpenNebula version is supported.''  
#Support for OCCI - EGI-approved access method
##Unfortunately this manual does not cover OpenNebula installation. You need to do this by yourself but this is well described here: [http://docs.opennebula.org/4.4/ http://docs.opennebula.org/4.4/]  
##''Described here: [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebul][https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula a] but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and which commands executed. <br>''
##Configure X.509 support according to [http://docs.opennebula.org/4.4/administration/authentication/x509_auth.html http://docs.opennebula.org/4.4/administration/authentication/x509_auth.html]  
#EGI User Authentication/Authorization
#Support for OCCI - EGI-approved access method  
##''You need to integrate with Perun. Described here [https://github.com/EGI-FCTF/fctf-perun https://github.com/EGI-FCTF/fctf-perun] but missing context of EGI Cloud site installation and missing commands to be executed.''
##''Described here: [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebul][https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula a] but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and which commands executed. <br>''  
#EGI Image Management
#EGI User Authentication/Authorization  
##''<span style="color: rgb(255,0,0);">Missing in the instructions for OpenNebula.</span> [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#VMcatcher Seems we have in https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#VMcatcher] section for Opennebula. Still no common context.''
##''You need to integrate with Perun. Described here [https://github.com/EGI-FCTF/fctf-perun https://github.com/EGI-FCTF/fctf-perun] but missing context of EGI Cloud site installation and missing commands to be executed.''  
#EGI Accounting
#EGI Image Management  
##''[https://github.com/EGI-FCTF/opennebula-cloudacc Described here: https://github.com/EGI-FCTF/opennebula-cloudacc] but missing context of EGI Cloud site installation - complete with info on which host the commands should be executed.''
##''<span style="color: rgb(255,0,0);">Missing in the instructions for OpenNebula.</span> [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#VMcatcher Seems we have in https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#VMcatcher] section for Opennebula. Still no common context.''  
#EGI Information System
#EGI Accounting  
##''[https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions Described here https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions] but again missing context of where these commands should be executed.''
##''[https://github.com/EGI-FCTF/opennebula-cloudacc Described here: https://github.com/EGI-FCTF/opennebula-cloudacc] but missing context of EGI Cloud site installation - complete with info on which host the commands should be executed.''  
#EGI Configuration Management Database (GOCDB)
#EGI Information System  
##''[https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions Described here https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions] but again missing context of where these commands should be executed.''  
#EGI Configuration Management Database (GOCDB)  
##''Manual not available. We need information on OpenNebula-specific service types to be registered in GOCDB.''
##''Manual not available. We need information on OpenNebula-specific service types to be registered in GOCDB.''


<span style="color: rgb(255,0,0);">What with support for CDMI in Opennebula?</span>
<span style="color: rgb(255,0,0);">What with support for CDMI in Opennebula?</span>  
 
== Synnefo ==
 
<span style="color: rgb(51,102,255);">''There are installation guides e.g. [https://www.synnefo.org/docs/synnefo/latest/install-guide-debian.html#install-guide-debian <span style="color: rgb(51,102,255);">https://www.synnefo.org/docs/synnefo/latest/install-guide-debian.html#install-guide-debian</span>] but it seems there is no EGI-specific installation guides.''</span>
==


== Synnefo  ==


<span style="color: rgb(51,102,255);">''There are installation guides e.g. [https://www.synnefo.org/docs/synnefo/latest/install-guide-debian.html#install-guide-debian <span style="color: rgb(51,102,255);">https://www.synnefo.org/docs/synnefo/latest/install-guide-debian.html#install-guide-debian</span>] but it seems there is no EGI-specific installation guides.''</span>
<br>


== Revision History  ==
== Revision History  ==

Revision as of 16:28, 28 January 2015

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home • Manuals • Procedures • Training • Other • Contact ► For: VO managers • Administrators



Title Setting up Cloud Resource Centre
Document link https://wiki.egi.eu/wiki/MAN10
Last modified 19 August 2014
Policy Group Acronym OMB
Policy Group Name Operations Management Board
Contact Group operations-support@mailman.egi.eu
Document Status DRAFT
Approved Date
Procedure Statement This manual provides information on how to set up Cloud Resource Centre.
Owner Owner of procedure



Introduction

EGI cloud supports 3 middlewares. It means you can base your cloud site installation on one of the following cloud software:

  • OpenNebula
  • OpenStack
  • Synnefo

If you want to install an EGI Cloud Site please have a look at our EGI Cloud Site Installation Manuals below.

Note:

EGI Cloud Site Installation Manual is a step-by-step instruction for

Cloud Site Admin. The manual is not meant to be a comprehensive on a topics related to the installation, it is a collection of steps taken by someone to install an EGI cloud site starting from a scratch. Commands

executed should be made available for someone to copy&paste and easily follow up. At some initial stage the manual may not cover all cases, but it is meant to be extended by other site admins while following up the manual. It is a living document.

The manuals

Current issues:

  • Documentation

for cloud components is written with assumption that the admin knows where (machine, neighbour components) this components should be installed. It is missing the general cloud site deployment context which


  • Documentation should address the prerequsities part. 
  • Documentation

should address the constraints and limitations part i.e. supported


  • Documentation should provide a contact person (per component) which can be contacted in case of questions/problems.  
  • Documentation should provide commands for checking validity of installation.


Prerequisities & Limitations

Whatever cloud stack you choose you need to prepare some things at the begining:

  1. Hardware (minimal hw requirements for small cloud site e.g up to 100 VMs):
    1. number of physical machines, performance/capacity requirements: RAM size
    2. disk space - how big, where must be connected, performance of network links (images are heavy!)
  2. DNS names, X.509 certificates
  3. Register in fedcloud VO
  4. Registration in AppDB to have access to private EGI VM image repository
  5. What operating systems are supported

OpenStack

EGI Cloud site can be based on OpenStack software with some EGI extensions. See deployment schema (Note: high level description on what modules are to be put on which machines.)

Installation steps:

  1. OpenStack installation
    1. Requirements'/lim'itations:
      1. Versions: Info on which OpenStack versions are supported by EGI on which operating system.  ' 
      2. Required by EGI configuration options for openstack: if there is anything required by EGI it should be written here: e.g. list of OpenStack components required by EGI, how to implement EGI security requirements on OpenStack, suggested network schema for Openstack - GRE or VXLAN.
        OpenStack installation you need to do by yourself following instructions here: http://docs.openstack.org/  
  2. Support for OCCI in OpenStack
    OCCI is the EGI-approved access method for computing resources that every cloud sites must expose.
    1. OCCI installation is described here: https://github.com/EGI-FCTF/occi-os/ but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and missing commands to be executed.
    2. Another nice installation guide (which one we should follow) is here  https://gilda.ct.infn.it/documents/26990/bee1363f-7444-4966-8cb2-f624e06542d6 do we want to provide people with two instructions?
  3. EGI User Authentication/Authorization
    Each cloud site must support users with X.509 certificates from fedcloud VO
    1. You need to have keystone-voms configured. http://ifca.github.io/keystone-voms/Missing context, missing command lines. Says it works for Grizzly, what with Havana, Icehouse?
  4. EGI Image Management
    Each cloud site must give access to EGI-approved VM images. An image clarifying functions and relations between vmcaster, vmcatcher, glance, glancepush and openstack handler for vmcatcher would be well welcome.
    1. Registration in AppDB to have access to private EGI VM image repository - missing, please describe steps to be done by Site Admin, should be as prerequisite steps because it requires manual step by someone
    2. VMCatcher - allows users to subscribe to VMs (unclear). https://github.com/hepix-virtualisation/vmcatcher
    3. Install EGI-customized version of glancepush: instruction: https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack software repo: https://appdb.egi.eu/store/software/python.glancepush/releases/0.0.x
    4. Install Openstack handler for vmcatcher: instruction: : https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#OpenStack software repo: https://appdb.egi.eu/store/software/openstack.handler.for.vmcatcher
  2. EGI Accounting
    Each cloud site must publish utilization data  EGI acccounting database which is APEL repository
    1. Instruction: https://github.com/EGI-FCTF/osssm/wiki
  3. EGI Information System
    Each cloud site must publish information to EGI information system which is based on BDII
    1. Instruction: https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions#OpenStack- site admin doing OpenStack do not want to read how it works for OpenNebula
  4. EGI Configuration Management Database (GOCDB)
    Each cloud site must register services in EGI configuration management database which is GOCDB
    Need information if cloud site must be separate from grid site or can be share. Shouldn't GOCDB step be earlier to allow nagios monitoring?
    1. Registering endpoints https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario5#GOCDB - ok, but we need info on what exactly has been registered in gocDB like "I have regiesterd os.acme.org as with type eu.egi.cloud.accounting.
    2. Registering SiteExtension Properties - is this still valid requirement? missing instruction
  5. Installation validaton - this is a new step: describe steps performed by site admin that confirms the site installation is working well according to EGI requirements. It

is better to have it as a separate, final step for all checks since a)

    1. Nagios step - missing
    2. check accounting - missing
    3. check vmcatcher subscription - missing
    4. check BDII publishing - missing
    5. check OCCI It is possible to reuse https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Compute_.28OCCI.29_checks
    6. check CDMI It is possible to reuse https://wiki.egi.eu/wiki/HOWTO04_Site_Certification_Manual_tests#Cloud_Storage_.28CDMI.29_checks
  2. Support for CDMI

I'd suggest to have separated part on CDMI and not mix it with OCCI.

OpenNebula

EGI Cloud site is based on OpenNebula software with some EGI extensions.See Deployment Schema (Note: here we need high level explanation on what modules are to be put on which machines.)

Stages of installation (similar for every middleware):

  1. OpenNebula installation with X.509 support
    1. Be consistent on requirements WHICH OpenNebula version is supported.
    2. Unfortunately this manual does not cover OpenNebula installation. You need to do this by yourself but this is well described here: http://docs.opennebula.org/4.4/
    3. Configure X.509 support according to http://docs.opennebula.org/4.4/administration/authentication/x509_auth.html
  2. Support for OCCI - EGI-approved access method
    1. Described here: https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:_Federated_AAI:OpenNebula but missing the context of EGI Cloud site installation i.e. what modules should be put on which machines, and which commands executed.
  3. EGI User Authentication/Authorization
    1. You need to integrate with Perun. Described here https://github.com/EGI-FCTF/fctf-perun but missing context of EGI Cloud site installation and missing commands to be executed.
  4. EGI Image Management
    1. Missing in the instructions for OpenNebula. Seems we have in https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Scenario8:Configuration#VMcatcher section for Opennebula. Still no common context.
  5. EGI Accounting
    1. Described here: https://github.com/EGI-FCTF/opennebula-cloudacc but missing context of EGI Cloud site installation - complete with info on which host the commands should be executed.
  6. EGI Information System
    1. Described here https://wiki.egi.eu/wiki/Fedclouds_BDII_instructions but again missing context of where these commands should be executed.
  7. EGI Configuration Management Database (GOCDB)
    1. Manual not available. We need information on OpenNebula-specific service types to be registered in GOCDB.

What with support for CDMI in Opennebula?

Synnefo

There are installation guides e.g. https://www.synnefo.org/docs/synnefo/latest/install-guide-debian.html#install-guide-debian but it seems there is no EGI-specific installation guides.

Revision History

Version Authors Date Comments
Retrieved from "https://wiki.egi.eu/w/index.php?title=MAN10&oldid=76659"