EGI Infrastructure operations oversight

EGI.eu Operations Oversight Pages

EGI Grid Operations oversight of the e-Infrastructure is a co-ordination task for ensuring that GRID monitoring across EGI runs smoothly. This team communicates among the 3 groups - Operations and e-Infrastructure Oversight (OE); Operational Documentation (OD); and "Coordination of interoperations between NGIs and with other Grids".

The Operations oversight team works with the Tool Developers (and particularly the OTAG group), NGIs and their Operations Teams (ROD). There are regular phone meetings for the co-ordinators and others working in the tasks. The OE co-ordinators also organise face to face meetings for the ROD teams 3 to 4 times a year.

Co-ordinators:

Ron Trompert (Chair), Marcin Radecki, Luuk Uljee

Deputy:

Malgorzata Krakowian

Contact:

• There are 3 mailing lists used for different cases:
  • manager-central-operator-on-duty AT mailman.egi.eu - for COD managerial issues like suggesting changes in procedures, tools. COD managers are recipients of this list.
  • central-operator-on-duty AT mailman.egi.eu - for reporting COD day-to-day issues like problems with tools or Nagios tests. COD shifters are recipients of this list.
  • all-central-operator-on-duty AT mailman.egi.eu - for contacting all ROD teams in NGIs. Each ROD team is a recipient of this list.

---

COD offical web pages

• Operations Support Metrics
• Operations tests list
• Availability and reliability tests list

Internal area

• Phone conference Meetings and Agenda

• ROD newsletter

• List of typical problems with RODs

COD shifters daily work instructions

In this section are collected all work instructions containing detailed information specifying exactly what steps are to be followed to carry out an activity.

Action	Description	Related procedures
GGUS tickets assigned to COD	COD shifter is obliged to check the current status of all GGUS tickets assigned to COD see Link to all GGUS tickets assigned to COD If the ticket is waiting for COD action then he/she should perform the action In case of a request for: ROD certification see New ROD team certification work instructions Creation of a new NGI see Creation of a new NGI process coordination In case where COD is also the Integration Process Coordinator, COD is responsible for the whole procedure. Operations Centre decommission see Operations Centre decommission process coordination COD validates the request and removes ROD information from all-operators mailing list Setting a Nagios test to an operations test see Procedure for setting a Nagios test to an operations test COD is responsible for coordinating the whole process. If the shifter doesn't know what kind of action should be taken, he/she should contact COD managers	New NGI creation process coordination Operations Centre decommission process coordination Procedure for setting Nagios test an operations test
Availability/reliability reports	Handling availability/reliability reports: Availability and reliability work instruction AR reports metrics	COD escalation procedure Availability and reliability monthly statistics procedure
Operational portal dashboard issues	COD dashboard link Escalation for operational problems with ROD - work instruction	COD escalation procedure
Handover	At the end of the shift a handover should be submitted (send to COD) via Handover tool in the Operational Portal Problems on the dashboard which will pass to next week: the ggus id of the ticket and when next escalation step should be taken GGUS tickets assigned to COD: for each ticket its last status and the action taken by the shifter should be provided Other issues: problems with tools etc.

NOTE: all procedures should contain the following template: https://wiki.egi.eu/wiki/PDT:Procedure_Template

OTAG topics

Operational Portal: Dashboard

• RT tickets
• COD interactions with Dashboard team (draft)
• COD topics to be discussed on OTAG meeting

GOC DB

• Collection of GOC DB requirements regarding COD work (draft)

Pages in draft state

• Improvements to Availability Calculation Procedure (draft)

• A/R fixing procedure (draft)