Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Agenda-2019-02-11"

From EGIWiki
Jump to navigation Jump to search
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Template:Op menubar}} {{Template:Doc_menubar}} {{TOC_right}}  
{{Template:Op menubar}} {{Template:Doc_menubar}} {{TOC_right}}  
[[Category:Grid Operations Meetings]]
[[Category:Grid Operations Meetings]]
= General information =
* EGI Conference 2019 https://www.egi.eu/news/egi-conference-2019-call-for-abstracts/
** Amsterdam, 6-8 May 2019
** call for abstracts https://indico.egi.eu/indico/event/4431/call-for-abstracts/
** Programme https://indico.egi.eu/indico/event/4431/program
** for operations, lots of hot topics; if you aim at contributing, please let us know! 


= Middleware  =
= Middleware  =
Line 6: Line 13:
== CREAM End Of Support Notice ==  
== CREAM End Of Support Notice ==  


https://community.egi.eu/t/cream-end-of-support-notice/438
* have a look at this topic in the EGI Community Forum https://community.egi.eu/t/cream-end-of-support-notice/438
* hot topic for the EGI Conference in May
* contribution from NGIs are welcome, to share concerns, solutions, problems, points of view


== Preview repository  ==
== Preview repository  ==
including in the next update: APEL Client/Server 1.8.0, davix 0.7.1, gfal2 2.16.0 and gfal2-python 1.9.4, FTS 3.8.1, xrootd 4.8.5


= Operations  =
= Operations  =
Line 60: Line 71:
*Under-performed sites in the past A/R reports with issues not yet fixed:
*Under-performed sites in the past A/R reports with issues not yet fixed:
**AfricaArabia: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139056
**AfricaArabia: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139056
***ZA-WITS-CORE: CE and SE unreachable
***ZA-WITS-CORE: CE and SE unreachable; waiting for a CREAM update to fix a Puppet configuration issue
**NGI_DE: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139058
**NGI_DE: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139058
***FZK-LCG2: CAs updated, A/R figures are improving
***FZK-LCG2: CAs updated, A/R figures are improving
***LRZ: GRAM5 failures
***LRZ: GRAM5 failures
**NGI_FI: https://ggus.eu/index.php?mode=ticket_info&ticket_id=138670
**NGI_FI: https://ggus.eu/index.php?mode=ticket_info&ticket_id=138670
***FI_TUT: problem with org.nordugrid.ARC-CE-IGTF which is returning "Script exited with code 1."
***FI_TUT: problem with org.nordugrid.ARC-CE-IGTF which is returning "Script exited with code 1."; waiting for the fix in the ARC probes.
**NGI_NL: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139059
**NGI_NL: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139059
***BEgrid-BELNET: bug in OpenNebula, as reported here https://ggus.eu/index.php?mode=ticket_info&ticket_id=138504
***BEgrid-BELNET: bug in OpenNebula, as reported here https://ggus.eu/index.php?mode=ticket_info&ticket_id=138504
Line 81: Line 92:
***UKI-SOUTHGRID-BHAM-HEP
***UKI-SOUTHGRID-BHAM-HEP
**ROC_Russia: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139507
**ROC_Russia: https://ggus.eu/index.php?mode=ticket_info&ticket_id=139507
***ITEP
***ITEP: issues during a major dCache version upgrade




suspended sites:
suspended sites:
== Decommissioning EMI WMS  ==
WMS servers can be decommissioned. Please '''follow the procedure [[PROC12]]'''. The plan is:
*Starting from January 2018, put the WMS servers in draining: this will block the submission of new jobs and will allow the jobs previously submitted to finish
**inform in advance your users that you are going to put in draining and then dismiss the WMS servers (as per [[PROC12]])
**there might be several VOs enabled on your WMS servers: in case only few of them need to use the service for few weeks more, you might disable the other VOs
*On Dec 14th EGI Operations sent a new broadcast to the VOs reminding the users the forthcoming WMS decommission
*From March, the nagios probe [https://wiki.egi.eu/wiki/MW_SAM_tests#General_tests eu.egi.sec.WMS] will return a CRITICAL status for the servers still in production, and the ROD teams will open a ticket to the sites that haven't finished the decommission process yet
'''[https://goc.egi.eu/portal/index.php?Page_Type=Downtimes_Calendar&view=month&certStatus=Certified&service_type=WMS WMS servers in downtime on GOC-DB]'''
[https://goc.egi.eu/portal/index.php?Page_Type=Services&serviceType=WMS&selectItemserviceType=WMS&ngi=&searchTerm=&production=TRUE&monitored=TRUE&certStatus=Certified&scopeMatch=all&servKeyNames=&servKeyValue= 5 WMS still registered on GOC-DB as production and monitored]
'''[https://argo-mon.egi.eu/nagios/cgi-bin/status.cgi?servicegroup=SERVICE_WMS&style=detail Status on nagios]'''
Tickets opened Oct 4th:
*BUDAPEST: https://ggus.eu/index.php?mode=ticket_info&ticket_id=137553 (SOLVED)
*WCSS64: https://ggus.eu/index.php?mode=ticket_info&ticket_id=137555 (SOLVED)
*CYFRONET-LGC2: https://ggus.eu/index.php?mode=ticket_info&ticket_id=137557 (to decide if replacing it with DIRAC or shut it down)
*AEGIS01-IPB-SCL: https://ggus.eu/index.php?mode=ticket_info&ticket_id=137558 (SOLVED)
*MA-01-CNRST: https://ggus.eu/index.php?mode=ticket_info&ticket_id=137559 (downtime expired, to check the WMS status)
*SARA-MATRIX: https://ggus.eu/index.php?mode=ticket_info&ticket_id=137560 (planned to set-up a national DIRAC server for replacing the WMS, Q1 2019)


== IPv6 readiness plans  ==
== IPv6 readiness plans  ==
Line 120: Line 107:
*webdav endpoints registered in GOC-DB: https://goc.egi.eu/gocdbpi/public/?method=get_service&&service_type=webdav  
*webdav endpoints registered in GOC-DB: https://goc.egi.eu/gocdbpi/public/?method=get_service&&service_type=webdav  
*link to nagios results: https://argo-mon.egi.eu/nagios/cgi-bin/status.cgi?servicegroup=SERVICE_webdav&style=detail
*link to nagios results: https://argo-mon.egi.eu/nagios/cgi-bin/status.cgi?servicegroup=SERVICE_webdav&style=detail
** '''9 CRITICAL out of 34 (OK status: 73.5%)'''
** '''5 CRITICAL out of 33 (OK status: 84.8%)'''
 
'''At the next OMB we are going to propose the inclusion of webdav probes in the critical profile'''


List of sites that not have completed the configuration yet:  
List of sites that not have completed the configuration yet:  
Line 126: Line 115:
*NGI_HR: '''egee.irb.hr''' https://ggus.eu/index.php?mode=ticket_info&ticket_id=134458 (webdav URL missing in GOC-DB...)
*NGI_HR: '''egee.irb.hr''' https://ggus.eu/index.php?mode=ticket_info&ticket_id=134458 (webdav URL missing in GOC-DB...)


List of sites that disabled webdav: UNIGE-DPNC, [https://ggus.eu/index.php?mode=ticket_info&ticket_id=131037 GR-01-AUTH], [https://ggus.eu/index.php?mode=ticket_info&ticket_id=131038 HG-03-AUTH], CETA-GRID, [https://ggus.eu/index.php?mode=ticket_info&ticket_id=131052 WUT]
Still some issues with:
*AEGIS01-IPB-SCL
*UKI-NORTHGRID-LANCS-HEP
*egee.srce.hr
*UNI-BONN


== Storage accounting deployment  ==
== Storage accounting deployment  ==
Line 151: Line 144:
*NGI_AEGIS: AEGIS04-KG https://ggus.eu/index.php?mode=ticket_info&ticket_id=131431
*NGI_AEGIS: AEGIS04-KG https://ggus.eu/index.php?mode=ticket_info&ticket_id=131431
*NGI_CH:
*NGI_CH:
**CSCS-LCG2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=131432 (it needs some help from dCache support)
**CSCS-LCG2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=131432 (dcache instructions has been updated, configuring STAR should be easier)
*NGI_DE:
*NGI_DE:
**DESY-HH https://ggus.eu/index.php?mode=ticket_info&ticket_id=131439 (postponed)
**DESY-HH https://ggus.eu/index.php?mode=ticket_info&ticket_id=131439 (postponed)
**GoeGrid https://ggus.eu/index.php?mode=ticket_info&ticket_id=131443
**GoeGrid https://ggus.eu/index.php?mode=ticket_info&ticket_id=131443
*NGI_IT:
*NGI_IT:
**INFN-ROMA1-CMS https://ggus.eu/index.php?mode=ticket_info&ticket_id=131482 (postponed)
**INFN-ROMA1-CMS https://ggus.eu/index.php?mode=ticket_info&ticket_id=131482 (new site-admin will work on it)
*NGI_NDGF: FI_HIP_T2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=131491
*NGI_NDGF: FI_HIP_T2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=131491 (SOLVED)


= AOB  =
= AOB  =

Latest revision as of 13:53, 11 February 2019

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home • Manuals • Procedures • Training • Other • Contact ► For: VO managers • Administrators


General information

Middleware

CREAM End Of Support Notice

Preview repository

including in the next update: APEL Client/Server 1.8.0, davix 0.7.1, gfal2 2.16.0 and gfal2-python 1.9.4, FTS 3.8.1, xrootd 4.8.5

Operations

ARGO/SAM

FedCloud

Feedback from DMSU

Notifications from ARGO about the nagios probes failures

In the process of implementing the notification system in ARGO, it was introduced the following changes in GOC-DB:

  • Notifications flag at the site level
  • Notifications flag at the service endpoint level

In this way ARGO will retrieve from GOC-DB the information about the sites and services whom sending the email notification and the related recipients.

The logic of the notifications is the following:

Site Service Notify?
Y Y Y
Y N N
N Y N
N N N

If there isn't any email contact defined at the service endpoint level, it will be used the site contact. Please review your contacts.

You can enable the notifications.

NOTE: It is not mandatory for the sites

Monthly Availability/Reliability


suspended sites:

IPv6 readiness plans

webdav probes in OPERATORS profile

The webdav probes was included in the ARGO_MON_OPERATORS profile after the approval in the January 2018 OMB: in this way the failures will generate an alarm on the dashboard, and the ROD teams can open a ticket to the failing sites. If no particular issue occurs, and if at least 75% of webdav endpoint are passing the tests, the probes will be added in the ARGO_MON_CRITICAL profile, so the results of these probes will be taken into account for the A/R figures.

At the next OMB we are going to propose the inclusion of webdav probes in the critical profile

List of sites that not have completed the configuration yet:

Still some issues with:

  • AEGIS01-IPB-SCL
  • UKI-NORTHGRID-LANCS-HEP
  • egee.srce.hr
  • UNI-BONN

Storage accounting deployment

During the September 2017 meeting, OMB has approved the full-scale deployment of storage accounting. The APEL team tested it with a group of early adopters sites, and the results prove that storage accounting is now production-ready.

Storage accounting is currently supported only for the DPM and dCache storage elements therefore only the resource centres deploying these kind of storage elements are requested to publish storage accounting data.

In order to properly install and configure the storage accounting scripts, please follow the instructions reported in the wiki: https://wiki.egi.eu/wiki/APEL/Storage

IMPORTANT: be sure to have installed the star-accounting.py script v1.0.4 (http://svnweb.cern.ch/world/wsvn/lcgdm/lcg-dm/trunk/scripts/StAR-accounting/star-accounting.py)

After setting up a daily cron job and running the accounting software, look for your data in the Accounting Portal: http://goc-accounting.grid-support.ac.uk/storagetest/storagesitesystems.html. If it does not appear within 24 hours, or there are other errors, please open a GGUS ticket to APEL who will help debug the process.

Test portal: http://accounting-devel.egi.eu/storage.php

IMPORTANT: Do not encrypt the storage records with your host certificate, please comment out the “server_cert” variable in sender.cfg

List of sites already publishing and of tickets opened is reported here.

Several sites are not publishing the storage accounting data yet. NGIs please follow-up with the sites the configuration of the script in order to speed-up the process.

AOB

Next meeting

Retrieved from "https://wiki.egi.eu/w/index.php?title=Agenda-2019-02-11&oldid=101055"