Difference between revisions of "FAQ Regional Operator on Duty"

From EGIWiki
Jump to: navigation, search
(How to handl issues during weekends and public holidays?)
(What to do with alarms when node is not in production and is part of production sit?)
Line 9: Line 9:
 
= What to do with alarms when node is not in production and is part of production sit?  =
 
= What to do with alarms when node is not in production and is part of production sit?  =
  
It often happens that testing nodes on production sites are set as non-production. In such case Nagios monitoring system will send information about all nodes, also about those not registered in GOCDB as production ones. As a result ROD will see on their dashboard alarms for non-production node. If it necessary to monitor such testing node it is recommended to put such non-production node in downtime.  
+
It often happens that testing nodes on production sites are set as non-production. In such case Nagios monitoring system will send information about all nodes. As a result ROD will see on their dashboard alarms for non-production node. If it necessary to monitor such testing node it is recommended to put such non-production node in downtime.
  
 
= What to do when a sites have multiple alarms/ticket?  =
 
= What to do when a sites have multiple alarms/ticket?  =

Revision as of 17:35, 24 October 2012

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security



How to handl issues during weekends and public holidays?

Due to the fact that weekends and public holidays are not considered working days it is noted that ROD teams do not have any responsibilities during these days. RODs should ensure that in these days tickets do not expire and alarms will not age above 72h.

What to do with alarms when node is not in production and is part of production sit?

It often happens that testing nodes on production sites are set as non-production. In such case Nagios monitoring system will send information about all nodes. As a result ROD will see on their dashboard alarms for non-production node. If it necessary to monitor such testing node it is recommended to put such non-production node in downtime.

What to do when a sites have multiple alarms/ticket?

When opening a ticket against a site with existing tickets ROD should consider that these problems may be linked or dependant on pending solutions. In such case ROD should use masking mechanism to gather and assigne alarms to one ticket rather than open a ticket for each alarm.

If the problem is different but maybe linked the expiry dates for each ticket should be synchronized to the latest date.

How to handle issues for site/node in downtime?

Handling tickets for site/node in downime

When a ticket has been raised against a site that subsequently enters downtime time, the expiry date on the ticket can be extended.

Sites that are in downtime will still have monitoring switched on and therefore may appear to be failing tests but no alarms on Operations Portal will be raised against them. ROD must take care that when opening tickets to ensure that they don't open tickets against sites in downtime.

Handling alarms for site/node in downime

It often happens that a failure occurred generating a lot of alarms and then site manager decided to put site in Downtime. Getting these alarms OK may take more than 72h when the issue is escalated to COD.
ROD should not create a ticket for sites/nodes in Downtime and is not obligated to deal with such alarms but it is recommended to close these alarms to avoid being escalated to COD. In such case as a reson of closing NON-OK alarm ROD should put link to the downtime in GOC DB.

Site in downtime for more than a month

If a site is in DOWNTIME for more than a month then it is advised that the site should go to the uncertified state.

What to do in case of accounting issue?

In case of problems with accounting it is not recommended to suggest downtime at the second step of the escalation process for this test. Accounting service is not a functionality which is critical for users but it still need to be follow up.