Agenda-05-12-2016

General information

• the Operations meeting will be on the 2nd Monday of the month
• the EGI Operations Meeting schedule for first half of 2016 is available on Indico: https://indico.egi.eu/indico/categoryDisplay.py?categId=32 and on the new summary page: https://wiki.egi.eu/wiki/Operations_Meeting

UMD/CMD/Preview

Preview repository

• Released on 2016-11-25:
  • Preview 1.6.0 AppDB info (sl6): ARC 15.03 update 10, frontier-squid 3.5.22-2.1, gfal2 utility 1.4.0, XRootD 4.5.0
  • Preview 2.6.0 AppDB info (CentOS 7): ARC 15.03 update 10, dCache 2.16.18, frontier-squid 3.5.22-2.1, gfal2 utility 1.4.0, XRootD 4.5.0, UI/WN tarball.

Note: EGI provides the preview repository without any additional quality assurance process, but the products are released as they are provided by the product team. EGI recommends the use of the UMD repositories, which contain software verified through the quality assurance process of UMD.

Operations

Software upgrades for OpenStack cloud RCs

• keystone-VOMS and cloud-info-provider updates available, need to be installed on all OpenStack sites
• as keystone-VOMS last version is only compatible with Liberty and Mitaka, in case OpenStack is Kilo (or older) an upgrade plan of OpenStack has been asked
• according to EGI policies https://wiki.egi.eu/wiki/PROC16_Decommissioning_of_unsupported_software OpenStack Kilo or older should NOT be running on the infrastructure! we are asking for discussing this point at the next OMB (October 27)
• as many sites are finding difficulties in planning upgrades against the very tight release cycle of OpenStack, please come with suggestion and reply with details in the tickets in order to shape the best (shared) proposal
• ticket campaign ONGOING for all OpenStack sites, asked to upgrade to keystone-VOMS >=8.0.3, cloud-info-provider >=0.6, and plans for the future (OpenStack version currently deployed, plans for upgrades, usual specific RC upgrade schedule), some results:
  • • TR-FC1-ULAKBIM FIXED, using FedCloud Appliance (updated version), using Liberty
    • IN2P3-IRES FIXED, using Liberty, upgrading to Mitaka
    • INFN-PADOVA-STACK FIXED, using Liberty, following Indigo indications about the version (staying with Liberty at the moment)
    • NCG-INGRID-PT, using Mitaka, up to date
    • IISAS-GPUCloud up to date, using Liberty, upgrading to Mitaka
    • FZJ using Kilo, not up to date, no plans for migrations due to user isolation bug https://ggus.eu/index.php?mode=ticket_info&ticket_id=121685 -> fixed in Newton, not in Mitaka (investigating about a backport to Mitaka)
    • BIFI using Grizzly, upgrading to Mitaka by mid of Nov
    • RECAS-BARI using Juno, no upgrades so far waiting for resolution of the user isolation bug, preparing upgrade to Mitaka from Ubuntu 16.04 LTS
    • IFCA-LCG2 FIXED, using Liberty
    • IISAS-FedCloud, downtime, upgrading to Mitaka from Ubuntu 16.04 LTS
    • SCAI FIXED, using Kilo, planning short term upgrade (no version specified yet)
    • CETA-GRID using Icehouse, planning mid-term upgrade
    • INDIGO-CATANIA-STACK, INFN-CATANIA-STACK, CYFRONET-CLOUD didn't reply yet
• feedback/issues reported from sites:
  • setup a pre-production environment that replicates the production environment and we use this testbed to implement/test/verify the upgrade procedure
  • suggested pilot RCs provide and share docs and guidelines when approaching to EOLs of CMFs (NCG-INGRID-PT)
  • New OpenStack versions are often buggy, need some grace time before upgrading
  • Unavailability of EGI Components for new versions
  • OpenStack shared with other projects and integrated with other tools
  • User identity auth issue still unsolved https://ggus.eu/index.php?mode=ticket_info&ticket_id=121685

Monthly Availability/Reliability

• November A/R figures not definitive yet

• Underperformed sites in the past A/R reports with issues not yet fixed:
  • NGI_DE GGUS 123836
    • TUDresden-ZIH: set-up a new CREAM-CE, the CA probes were failing. Issues on SRM service
  • NGI_IBERGRID: GGUS 124371
    • CIEMATIC-TIC: decommissioning the storage element
  • NGI_NL: GGUS 123532
    • BelGrid-UCL: UNKNOWN status returned by CREAM probes, asked a recomputation

• Sites suspended after October A/R report:
  • UNI-DORTMUNT (NGI_DE): downtime longer than a month
  • TW-eScience (AsiaPacific): long standing issues
  • INFN-CAGLIARI (NGI_IT): underperorming since too many months, never recovered

ARGO proposal to use GOCDB as the only source of topology information

• slides in October Operations Meeting agenda
• ARGO Proposal (September OMB)
• ARGO and GOC-DB updates from November OMB

• Timescale:
  • New GOC-DB release on Dec 7th including a boolean ‘monitored’ flag for the service endpoints
  • Then creation of a web UI view for uncertified sites in ARGO
  • Uncertified sites will be asked to fill in the service endpoints information. Follow the How to add URL service endpoint information into GOC-DB
    • (OPTIONAL) use the GOC-DB test instance for testing the procedure
  • As information is added in the GOCDB, uncertified sites/services will be picked up by the ARGO Monitoring Engine and they will start to be monitored
  • By Q2 2017: support for multiple service endpoints

VAPOR

• VAPOR 2.1 released in September, it replaces GSTAT
• important for presenting the amount of computing and storage resources of the infrastructure
• each NGI should review the information provided by their sites and let us know any inconsistency: http://operations-portal.egi.eu/vapor/resources/GL2ResSummary
  • we need your feedback to improve the service
  • some known issues will be fixed in the next release
• new version 2.2 is about to be released:
  • please test it going on the dev instance http://operations-portal.egi.eu/vapor_dev
  • report any comment into https://ggus.eu/index.php?mode=ticket_info&ticket_id=124872

AOB

Next meeting

• Jan 9th, 2016 https://indico.egi.eu/indico/event/3139/
• new calendar available until June 2017 https://indico.egi.eu/indico/category/32/