Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Fedcloud-tf:WorkGroups:Scenario5"

From EGIWiki
Jump to navigation Jump to search
(Redirected page to Federated Cloud Monitoring)
 
(107 intermediate revisions by 8 users not shown)
Line 1: Line 1:
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}
#REDIRECT[[Federated_Cloud_Monitoring]]
 
== Scenario 5: Reliability/Availability of Resource Providers  ==
 
<font color="red">Leader: Daniele Cesini, INFN</font>
 
== Scenario collaborators  ==
 
{| border="1"
|-
! Role
! Institution
! Name
|-
| Scenario leader
| INFN
| Daniele Cesini
|-
|
Collaborator
 
| SRCE<br>
| &nbsp; Emir Imamagic
|-
|
Collaborator
 
| CESGA
| Ivan Diaz
|-
| Collaborator
| CESGA
| Alvaro Simon
|}
 
 
 
== What Monitoring means in this context  ==
 
Monitoring in this context is the monitoring of the availability (and reliability) of the cloud resorces provided by the resource provided. What will be tested is the possibility for an hipotetical user to instantiate at least one pre-defined virtual machine within a given period of time. It consists of an "external" monitoring, no data will be collected from inside the VMs. Monitoring the capabilities of the cloud resource providers in terms of how many resources are available is beyond the scope of this Scenario, at least in its initial phase.Possible evolution of the FedCloud monitoring will be evaluated when the basic monitoring will be in place.<br>
 
<br>
 
The outcome of Scenario5 will be a system that is able to run at least one probe on each&nbsp; Resource Provider paticipating to the FedCloud.
 
The probe will have an OK status if the VM can be instantiated.
 
<br>
 
Given the experience accumulated with the [http://www.nagios.org NAGIOS] system within the EMI and EGI projects the monitoring framework will be based on NAGIOS. This has also the advantage to ease the integration of the FedCloud monitoring framework in the [https://wiki.egi.eu/wiki/SAM SAM]&nbsp;monitoring sytem used by the EGI project to monitor the production infrastructure.
 
<br>
 
Status history of all the probes will be available trough web interfaces.
 
<br>
 
<br>
 
== The proposed approach  ==
 
The proposed approach is to have a central monitoring instance that run probes on al the FedCloud resources and collects their output.
 
The central instance could be a full blown SAM&nbsp;instance as those available through EGI ([https://wiki.egi.eu/wiki/SAM_Instances https://wiki.egi.eu/wiki/SAM_Instances]) or a simple NAGIOS box. This has to be decided.
 
<br>
 
The following steps need to be completed in order to have the approach established.
 
1. '''Identify the Central instance:''' One resource or technology provider needs to provide a machine (virtual or real) where the testing instance would be deployed. Based on our experience providing 1GB RAM, 1 CPU/core and at least 10GB of disk is sufficient. This instance will be used to monitor all the clouds provided by resource providers. EGI-JRA1 will help with installation of this SAM instance.
 
<br>
 
2. '''Creation of basic probes:''' for each technology/Resource provider a basic probe should be created to test:
 
*login functionality to the remote VM web interface (assuming that such web interface is available)&nbsp;
*pre-defined VM instantiation
 
The idea is to start deinfing a skeleton probe with one of the technology providers that will be used to create all other probes.
 
The characteristics of the pre-defined VM are to be discussed with TPs
 
this step can be split in sub steps
 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2.a Identify a technology provider that voulunteer to create the skeleton probe with the help of EG-JRA1
 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2.b Use the skeleton probe to create probes for all the other TPs
 
<br>
 
3.'''Integration of probes: '''Once all the probes are created they can be integrated into the NAGIOS or SAM&nbsp;instance that can start collecting status data about the FedCloud
 
<br>
 
We estimante that these 3 initial steps can be accomplished before the '''end of February 2012'''.
 
<br>
 
== Possible evolution  ==
 
1. '''Integration in the EGI production infrastructure''': Once all the probes are provided and tested on the dedicated central FedCloud instance we can start the discussion on integration with EGI operations, two options are foreseeable:
 
*keep a single instance for all resources
*distribute various monitoring instances over the current NGI model.
 
The next steps are integration with other operational tools used in EGI (operations portal, GOCDB, etc.)
 
<br>
 
2. '''Improvement of probes:''' probes can be developed based on APIs (OCCI or other) and can&nbsp; will perform actual capability (storage, computing) tests and can provide more accurate information on status of individual resource provider. This step depends on other activities (e.g. definition of common interface) and previous steps of scenario5.<br>
 
<br>
 
== NAGIOS Probes  ==
 
'''Who has the responsibility to develop probes?''' Following the EGI model probes are developed by the Technology Providers and are integrated into the monitoring framework by the EGI-JRA1 staff that can also provide support during the initial phase of probes development in order to give guidelines and templates.
 
'''Informationon''' on how to develop NAGIOS probes can be retrieved in the [https://tomtools.cern.ch/confluence/display/SAMDOC/Probes+Development+Policy SAM Developmes Guide]
 
'''List of available probes '''within EGI is reported in the [https://tomtools.cern.ch/confluence/display/SAMDOC/Released+Probes SAM Administrrator Guide]
 
<br>
 
== The SAM&nbsp;System  ==
 
== Scenario5 meetings  ==
 
== Scenario5 Actions  ==
 
Scenario5 Action available on [https://oerc.basecamphq.com/projects/7732005-egi-federated-cloudstask-force/todo_lists/16450232?source=sidebar_link RAL Basecamp]
 
== Further Resources  ==
 
[[SAM|The SAM system EGi wiki pages]]
 
[[Image:Flessr nagios probes.pdf|FleSSR Nagios Cloud Probes Document]] (Thanks to David Wallom)

Latest revision as of 12:09, 8 June 2015