Agenda-2020-05-11

From EGIWiki
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators


Back to https://wiki.egi.eu/wiki/Operations_Meeting

General information

Middleware

UMD


CMD

Preview repository

  • released on 2020-05-08
    • Preview 1.27.0 AppDB info (sl6): ARC 6.5.0 and 6.6.0, CVMFS 2.7.2, dCache 5.2.20, frontier-squid 4.11.2, gfal2 2.17.2, xrootd 4.11.3
    • Preview 2.27.0 AppDB info (CentOS 7): ARC 6.5.0 and 6.6.0, CVMFS 2.7.2, dCache 5.2.20, frontier-squid 4.11.2, gfal2 2.17.2, xrootd 4.11.3

Operations

ARGO/SAM

When successful:

CREAM JobOutput OK: retrieved outputSandbox: ['std.err', 'std.out']

**** std.err ****
+ versionFilter='ή\._PIPE_ί\._PIPE_έ\._PIPE_ΰ\.'
+ type=unknow
+ mwver=error
+ '[' -f /etc/umd-release ']'
+ type=UMD
++ cat /etc/umd-release
++ awk '{print $3}'
+ mwver=4.1.3
+ set +x


**** std.out ****
atlaswn184 has UMD 4.1.3

When it fails:

CREAM JobOutput ERROR [DONE-OK, exitCode=1 ]: retrieved outputSandbox: ['std.err', 'std.out']

**** std.err ****
+ versionFilter='ή\._PIPE_ί\._PIPE_έ\._PIPE_ΰ\.'
+ type=unknow
+ mwver=error
+ '[' -f /etc/umd-release ']'
+ '[' -f glite-version ']'
+ '[' -f /etc/emi-version ']'
+ '[' -f lcg-version ']'
++ hostname -s
+ echo 'ERROR: [glite_PIPE_lcg_PIPE_emi]-version was not found in n172'
+ exit 1


**** std.out ****
ERROR: [glite_PIPE_lcg_PIPE_emi]-version was not found in n172

FedCloud

Feedback from DMSU

Monthly Availability/Reliability

IPv6 readiness plans

ARC Middleware 5 end of support, migration to ARC 6

  • No new feature development is planned or going on for ARC5 and no bug-fixing development will happen on ARC5 code base in the future except for security issues.
  • Security fixes for ARC5 will be provided till end of June 2020.
  • Production Sites already running ARC 5 will be able to get deployment and configuration troubleshooting help via GGUS till end June 2021. This we call "operational site support".
  • ARC5 is available in EPEL7 and will stay there. EPEL8 will only contain ARC 6.


$ ldapsearch -x -LLL -H ldap://egee-bdii.cnaf.infn.it:2170 -b "GLUE2GroupID=grid,o=glue" '(&(objectClass=GLUE2Endpoint)(&(GLUE2EndpointImplementationName=nordugrid-arc)(GLUE2EndpointTechnology=gridftp)))' GLUE2EndpointImplementationVersion GLUE2EndpointID | grep GLUE2EndpointImplementationVersion | sort | uniq -c
      1 GLUE2EndpointImplementationVersion: 20190403020701
      1 GLUE2EndpointImplementationVersion: 20200217020714
      1 GLUE2EndpointImplementationVersion: 20200226020744
      1 GLUE2EndpointImplementationVersion: 20200228020731
      1 GLUE2EndpointImplementationVersion: 20200424020718
      1 GLUE2EndpointImplementationVersion: 20200505020715
      1 GLUE2EndpointImplementationVersion: 4.1.0
      1 GLUE2EndpointImplementationVersion: 5.0.2
      1 GLUE2EndpointImplementationVersion: 5.1.3
      1 GLUE2EndpointImplementationVersion: 5.3.0
      5 GLUE2EndpointImplementationVersion: 5.3.1
      6 GLUE2EndpointImplementationVersion: 5.4.1
     11 GLUE2EndpointImplementationVersion: 5.4.2
      7 GLUE2EndpointImplementationVersion: 5.4.3
     47 GLUE2EndpointImplementationVersion: 5.4.4
      2 GLUE2EndpointImplementationVersion: 6.2.0
      5 GLUE2EndpointImplementationVersion: 6.4.1
     14 GLUE2EndpointImplementationVersion: 6.5.0
      5 GLUE2EndpointImplementationVersion: 6.6.0

LCGDM end of support and migration to / enabling of DOME

  • Deployment statistics (May 8th):
$ ldapsearch -x -LLL -H ldap://egee-bdii.cnaf.infn.it:2170 -b "GLUE2GroupID=grid,o=glue" '(&(objectClass=GLUE2Manager)(GLUE2ManagerProductName=DPM))' GLUE2ManagerProductVersion GLUE2ManagerID | grep GLUE2ManagerProductVersion | sort | uniq -c
     1 GLUE2ManagerProductVersion: 1.10.0
    66 GLUE2ManagerProductVersion: 1.13.0
     2 GLUE2ManagerProductVersion: 1.13.1
    12 GLUE2ManagerProductVersion: 1.13.2
     3 GLUE2ManagerProductVersion: 1.8.10
     1 GLUE2ManagerProductVersion: 1.8.8
     1 GLUE2ManagerProductVersion: 1.8.9
     4 GLUE2ManagerProductVersion: 1.9.0


Liasing with WLCG to follow-up the upgrade. Opened GGUS tickets asking the following:

  • all the sites with older DPM versions than 1.12 are suggested to upgrade to the latest DPM version , following the guide DPM upgrade (chapter 1 Upgrade to DPM 1.10.0 "Legacy Flavour" and chapter 2 Upgrade to DPM 1.10.0 "Dome Flavour")
    • DOME and the old LCGDM (srm protocol) will coexist
  • Monitoring: sites should enable the monitoring of the HTTP/WebDav and/or GridFTP endpoints
    • register the storage service endpoint as webdav and/or globus-GRIDFTP service type, with production flag disabled, providing respectively the URL field and the Extension Properties information as explained in the HOWTO21
    • check if the tests are ok
    • switch the production flag to "yes"

List of tickets


Site Ticket Notes
AEGIS03-ELEF-LEDA https://ggus.eu/index.php?mode=ticket_info&ticket_id=143152 SE marked as not production due to some issues that need to be fixed
GRID-UNAM https://ggus.eu/index.php?mode=ticket_info&ticket_id=143161 scheduled a downtime in Feb for upgrade
INDIACMS-TIFR https://ggus.eu/index.php?mode=ticket_info&ticket_id=142245 upgrade delayed to to COVID-19 lockdown in Mumbai
OBSPM https://ggus.eu/index.php?mode=ticket_info&ticket_id=143169 they asked for quattor documentation...
TASK https://ggus.eu/index.php?mode=ticket_info&ticket_id=143174 problem with starting xrootd...
UA_ICYB_ARC https://ggus.eu/index.php?mode=ticket_info&ticket_id=143178
WCSS64 https://ggus.eu/index.php?mode=ticket_info&ticket_id=143182 they need some time to gain enough knowledge for doing the upgrade....
GR-07-UOI-HEPLAB https://ggus.eu/index.php?mode=ticket_info&ticket_id=143467 still on slc6, some problems with the upgrade; before the end of the year all the site will be migrated to CentOS 7
Hephy-Vienna https://ggus.eu/index.php?mode=ticket_info&ticket_id=143277 DPM will be replaced with EOS during Q1 2020
HK-LCG2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=143471 April 2020
ICM https://ggus.eu/index.php?mode=ticket_info&ticket_id=143091 dpm in the newest version. Now we are setting quota tokens...
IN2P3-IPNL https://ggus.eu/index.php?mode=ticket_info&ticket_id=143082 migration to EOS, dpm should be dismissed by mid 2020
IN2P3-IRES https://ggus.eu/index.php?mode=ticket_info&ticket_id=143070 testing the upgrade on the test infrastructure
NIKHEF-ELPROD https://ggus.eu/index.php?mode=ticket_info&ticket_id=143286 We won't upgrade. we plan to migrate to dCache before the end of 2019.
PSNC https://ggus.eu/index.php?mode=ticket_info&ticket_id=143474
ru-PNPI https://ggus.eu/index.php?mode=ticket_info&ticket_id=143281
UKI-SCOTGRID-DURHAM https://ggus.eu/index.php?mode=ticket_info&ticket_id=143465 migration to CentOS7, first....
UKI-SCOTGRID-ECDF https://ggus.eu/index.php?mode=ticket_info&ticket_id=143077 upgrading to CentOS7 first without DOME
UKI-SCOTGRID-GLASGOW https://ggus.eu/index.php?mode=ticket_info&ticket_id=143076 plan in progress
UKI-SOUTHGRID-BRIS-HEP https://ggus.eu/index.php?mode=ticket_info&ticket_id=143083


SECMON failures

Several CEs are failing the job submission tests, preventing pakiti to check the vulnerabilities fixes on the WNs.

AOB

Next meeting

June 8th, 2020 https://indico.egi.eu/event/4900/