Resource Centres OLA and Resource infrastructure Provider OLA reports

Introduction

EGI availability and reliability statistics are produced every month for all certified production sites. The current version of the Site-NGI Operational Level Agreement defines the following requirements:

• minimum tolerated availability: 70%,
• minimum tolerated reliabilty: 75%.

For each monthly report, underperforming sites are requested through a GGUS to motivate the poor performance provided.

Suspension procedure: sites which have an availability of less than 50% for three consecutive months will be suspended, i.e. removed from the production infrastructure.

Statistics

• July 2010 [1]
• June 2010 [2]
• May 2010 [3]
• January 2008 - April 2010 (EGEE league tables)

Status of availability followup procedure and suspended sites

• Status of availability followup procedure
• List of suspended sites (2009)

Tools and documentation

• xls availability/reliability report generator (providing access to the database including Nagios results for OPS and SAM results for VOs)
• Availability/reliability computation algorithm
• Availability Computation Engine (ACE)
• OLD: EGEE-III Comments on site availability and reliability statistics

Note: in EGI sites do not need to provide comments anymore. In EGI comments will be solicited and collected through GGUS tickets instead.

Description of the process

• Generation of statistics

Availability and reliability statistics are automatically generated the first week of the month by GridView in pdf format and placed under [4]. An Excel version is available at [5]

• Preliminary processing

Once the reports are generated, sanity checks are performed by EGI SA1 (Task TSA1.8). After this step is completed, statistics are uploaded into the EGI document server. Links to monthly statistics will be provided on a regular basis at this wiki page.

• Publication

An announcement of the new results is distributed by EGI SA1 (TSA1.8) to the NGI Operations Managers mailing list. COD (TSA1.7) is responsible of supervising statistics by chasing NGIs to chase sites that need to provide comments in case thresholds are not met, and identifies sites eligible for suspension. This phase starts by filing a ticket to the COD Support Unit. The overall comments gathering process is handled through tickets.

• Handling of sites below targets

For a site that misses availability/reliability targets but is not eligible for suspension:

1. a child ticket is opened by the COD team and assigned to the respective NGI, asking for explanation to be given
2. the explanation must be produced within 10 working days since the ticket is received by the site (please see known issues section [6])
3. if the explanation is found satisfactory the ticket is closed
4. conversely if the explanation is not given in due time, or the explanation is found inadequate, COD escalation procedure will be followed, which at this time is still being revised and is tracked into EGI RT [7]
5. the child ticket can then be closed
6. the parent ticket will be closed when all child tickets have been closed.

• Handling of sites that are eligible for suspension

For a site that is eligible for suspension:

1. a child ticket is opened by the COD team assigned to appropriate NGI, notifying that the site will be suspended within 10 working days (please see known issues section [8])
2. after the 10 days period passes, the site is suspended by COD unless the NGI has intervened or the EGI Chief Operations officer objects
3. in the case of NGI intervention, non suspension will occur if both the COD and COO agree on the reasoning provided by the NGI
4. the child ticket closes either when the site is suspended or when suspension is canceled
5. the parent ticket will be closed when all child tickets have been closed

• Wiki follow up page

Sites that fail to provide explanations justifying the failure to meet OLA targets, or the explanation is found inadequate, as well as sites that are suspended, will be recorded in a wiki page [9]

Known issues and recommendations to NGIs

1. When reprocessing data, GridView takes into account the Certification status of the site at that moment and not the full Certification status history. Thus newly certified sites will get inaccurate Availability/Reliability figures for the month they were certified. Because the Certification status history is not currently available in the operations tools, until a solution is implemented NGIs should check if they have sites affected by this issue and report it as explanation. More information at [10]