Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Agenda-05-12-2016"

From EGIWiki
Jump to navigation Jump to search
 
(18 intermediate revisions by 2 users not shown)
Line 7: Line 7:


= UMD/CMD/Preview  =
= UMD/CMD/Preview  =
* '''UMD 3.14.6''' http://repository.egi.eu/2016/11/23/release-umd-3-14-6/
** Added lcmaps-plugins-vo-ca-ap for SL6, needed for supporting the IGTF IOTA profile of CAs
* '''UMD 4.3.0''' 'October' released http://repository.egi.eu/2016/11/10/release-umd-4-3-0/
** ARC, GFAL2, XROOT, Davix, dCache, ARGUS, Gridsite, edg-mkgrid, umd-release for CentOS7
** ARC, GFAL2, XROOT, Gridsite, edg-mkgrid, umd-release, GRAM5, DPM, Globus GridFTP, globus-default-security, MyProxy, Davix, dCache, VOMS, YAIM core, lcas-lcmaps for SL6
** '''update: UMD 4.3.1''' http://repository.egi.eu/2016/11/23/release-umd-4-3-1/
** CentOS7: lcas-lcmaps-gt4-interface 0.3.0-0.3.1, lcmaps 1.6.6, lcas 1.3.19, glExec 1.2.3, glExec-WN 1.3.0, lcmaps-plugins 1.7.1
** SL6: ARGUS server 1.7
* next regular release (4.4.0) on February: https://wiki.egi.eu/wiki/UMD_Release_Schedule
* '''UMD 3.14.7 and UMD 4.3.2 RCs are ready''', will be released today/tomorrow
** permanent fix for umd-release issue (fixing both new installations and upgrades)
* Comparison between UMD3/SL6 and UMD4/SL6 available, to be improved: https://wiki.egi.eu/wiki/UMD3_UMD4_products
* '''CMD-OS (OpenStack Mitaka)''' to be released (delayed after deadline of Nov29)
** Operating systems: CentOS7, Ubuntu14 (but Ubuntu14 distribution works also with Ubuntu16)
** Product ID Cards listed here https://wiki.egi.eu/wiki/EGI_Cloud_Middleware_Distribution_products
** Queue available to all-sso: https://rt.egi.eu/rt/Dashboards/10379/Software%20Provisioning%20CMD-OS-1
** Products to be included in this release
** Under verification: cASO, rOCCI CLI,
** Ready/almost ready: keystone-VOMS 9.0.3, ooi (OCCI 0.3.2-1), GridSite, BDII Information Provider
** First Indigo-datacloud products to be included (nova-docker, java-reposync), missing packages
** More products will be included in an update (December/January)
* '''CMD-ONE (OpenNebula 4 version)''' just started, will include products from Indigo as well


== Preview repository ==
== Preview repository ==
Line 17: Line 46:


= Operations =
= Operations =
== Decommissioning of mon.egi.eu ==
* tomorrow, December 6th 2016 ARGO team will decommission the old SAM GridMon box mon.egi.eu, housing central ATP and POEM. These services became obsolete when we switched to central monitoring instances in July.
** VO SAM instances will not be affected as they are using local ATP and POEM.
** Remaining NGI SAM instances rely on central ATP and will no longer get topology updates, so this gives you extra incentive to decommission them.
== cloudmon.egi.eu dismissed ==
* new central instances are available
** https://argo-mon.egi.eu/nagios/
** https://argo-mon2.egi.eu/nagios/


== Software upgrades for OpenStack cloud RCs ==
== Software upgrades for OpenStack cloud RCs ==
Line 24: Line 65:
* according to EGI policies  https://wiki.egi.eu/wiki/PROC16_Decommissioning_of_unsupported_software '''OpenStack Kilo or older should NOT be running on the infrastructure!''' we are asking for '''discussing this point at the next OMB (October 27)'''  
* according to EGI policies  https://wiki.egi.eu/wiki/PROC16_Decommissioning_of_unsupported_software '''OpenStack Kilo or older should NOT be running on the infrastructure!''' we are asking for '''discussing this point at the next OMB (October 27)'''  
* as many sites are finding difficulties in planning upgrades against the very tight release cycle of OpenStack, '''please come with suggestion and reply with details in the tickets''' in order to shape the best (shared) proposal
* as many sites are finding difficulties in planning upgrades against the very tight release cycle of OpenStack, '''please come with suggestion and reply with details in the tickets''' in order to shape the best (shared) proposal
* '''ticket campaign ONGOING for all OpenStack sites''', asked to upgrade to keystone-VOMS >=8.0.3, cloud-info-provider >=0.6, and plans for the future (OpenStack version currently deployed, plans for upgrades, usual specific RC upgrade schedule), some results:  
* '''ticket campaign ONGOING for all OpenStack sites''', asked to upgrade to keystone-VOMS >=8.0.3, cloud-info-provider >=0.6, and plans for the future (OpenStack version currently deployed, plans for upgrades, usual specific RC upgrade schedule), '''UPDATE''':  
*** TR-FC1-ULAKBIM FIXED, using FedCloud Appliance (updated version), using Liberty
** INDIGO-CATANIA-STACK and INFN-CATANIA-STACK moving to Mitaka (no plan)
*** IN2P3-IRES FIXED, using Liberty, upgrading to Mitaka
** IISAS-GPUCloud Liberty
*** INFN-PADOVA-STACK FIXED, using Liberty, following Indigo indications about the version (staying with Liberty at the moment)
** FZJ user isolation bug fixed in Newton, not in Mitaka (investigating about a backport to Mitaka), waiting for solution
*** NCG-INGRID-PT, using Mitaka, up to date
** IN2P3-IRES Mitaka
*** IISAS-GPUCloud up to date, using Liberty, upgrading to Mitaka
** CETA-GRID using Icehouse, planning mid-term upgrade (Newton?)
*** FZJ using Kilo, not up to date, no plans for migrations due to user isolation bug https://ggus.eu/index.php?mode=ticket_info&ticket_id=121685 -> fixed in Newton, not in Mitaka (investigating about a backport to Mitaka)
** IISAS-FedCloud Mitaka from Ubuntu 16.04 LTS installed
*** BIFI using Grizzly, upgrading to Mitaka by mid of Nov
** BIFI upgrading to Mitaka
*** RECAS-BARI using Juno, no upgrades so far waiting for resolution of the user isolation bug, preparing upgrade to Mitaka from Ubuntu 16.04 LTS
** SCAI upgraded to Mitaka
*** IFCA-LCG2 FIXED, using Liberty
** INFN-PADOVA-STACK FIXED using Liberty
*** IISAS-FedCloud, downtime, upgrading to Mitaka from Ubuntu 16.04 LTS
** IFCA-LCG2 using Liberty
*** SCAI FIXED, using Kilo, planning short term upgrade (no version specified yet)
** CYFRONET-CLOUD running Juno, evaluating Mitaka
*** CETA-GRID using Icehouse, planning mid-term upgrade
** TR-FC1-ULAKBIM FIXED using Liberty
*** ''INDIGO-CATANIA-STACK, INFN-CATANIA-STACK, CYFRONET-CLOUD didn't reply yet''
** NCG-INGRID-PT, using Mitaka, up to date
* '''feedback/issues reported from sites''':
** RECAS-BARI preparing upgrade to Mitaka from Ubuntu 16.04 LTS (deadline by  end of year)
** setup a pre-production environment that replicates the production environment and we use this testbed to implement/test/verify the upgrade procedure
** 100IT Liberty (evaluating Mitaka)
** suggested pilot RCs provide and share docs and guidelines when approaching to EOLs of CMFs (NCG-INGRID-PT)
** New OpenStack versions are often buggy, need some grace time before upgrading
** Unavailability of EGI Components for new versions
** OpenStack shared with other projects and integrated with other tools
** User identity auth issue still unsolved https://ggus.eu/index.php?mode=ticket_info&ticket_id=121685


== Monthly Availability/Reliability ==
== Monthly Availability/Reliability ==
Line 53: Line 89:
** '''NGI_DE''' [https://ggus.eu/index.php?mode=ticket_info&ticket_id=123836 GGUS 123836]
** '''NGI_DE''' [https://ggus.eu/index.php?mode=ticket_info&ticket_id=123836 GGUS 123836]
***TUDresden-ZIH: set-up a new CREAM-CE, the CA probes were failing. Issues on SRM service
***TUDresden-ZIH: set-up a new CREAM-CE, the CA probes were failing. Issues on SRM service
**'''NGI_IT''': [https://ggus.eu/index.php?mode=ticket_info&ticket_id=123531 GGUS 123531]
***INFN-CAGLIARI: underperforming for more than 3 months, no further feedback provided, eligible to suspension
**'''NGI_IBERGRID''': [https://ggus.eu/?mode=ticket_info&ticket_id=124371 GGUS 124371]
**'''NGI_IBERGRID''': [https://ggus.eu/?mode=ticket_info&ticket_id=124371 GGUS 124371]
***CIEMATIC-TIC: decommissioning the storage element
***CIEMATIC-TIC: decommissioning the storage element
Line 63: Line 97:
**UNI-DORTMUNT (NGI_DE): downtime longer than a month
**UNI-DORTMUNT (NGI_DE): downtime longer than a month
**TW-eScience (AsiaPacific): long standing issues
**TW-eScience (AsiaPacific): long standing issues
**INFN-CAGLIARI (NGI_IT): underperorming since too many months, never recovered


== ARGO proposal to use GOCDB as the only source of topology information ==
== ARGO proposal to use GOCDB as the only source of topology information ==
Line 68: Line 103:
* [https://indico.egi.eu/indico/event/3006/material/slides/0.pdf slides in October Operations Meeting agenda]
* [https://indico.egi.eu/indico/event/3006/material/slides/0.pdf slides in October Operations Meeting agenda]
* [https://indico.egi.eu/indico/event/2810/contribution/3/material/0/ ARGO Proposal (September OMB)]
* [https://indico.egi.eu/indico/event/2810/contribution/3/material/0/ ARGO Proposal (September OMB)]
* The plan:
* [https://indico.egi.eu/indico/event/2814/contribution/6/material/slides/ ARGO] and [https://indico.egi.eu/indico/event/2814/contribution/7/material/slides/ GOC-DB] updates from November OMB
**Develop new features on GOC-DB and ARGO:
 
***GOC-DB: create the new�boolean attribute "Monitored" on the ServiceEndpoints
*Timescale:
***ARGO: change the GOCDB connectors in order to take into account also this attribute
**New GOC-DB release on Dec 7th including a boolean ‘monitored’ flag for the service endpoints
**Start with the uncertified sites, then all the others
**Then creation of a web UI view for uncertified sites in ARGO
**Interim period: consuming both BDII and GOC-DB
**Uncertified sites will be asked to fill in the service endpoints information. Follow the [https://wiki.egi.eu/wiki/HOWTO21 How to add URL service endpoint information into GOC-DB]
***If no issues, only GOC-DB will be kept as topology source
***('''OPTIONAL''') use the [https://gocdb-test.esc.rl.ac.uk/portal/index.php GOC-DB test instance] for testing the procedure
* Proposed timescale:
**As information is added in the GOCDB, uncertified sites/services will be picked up by the ARGO Monitoring Engine and they will start to be monitored
**November
** By Q2 2017: support for multiple service endpoints
***Complete developments on the ARGO/GOCDB sides
***New report will be configured on ARGO for uncertified sites
**December:
***Uncertified sites will be requested to update their information on GOCDB
***As information is added in the GOCDB, uncertified sites/services will be picked up by the ARGO Monitoring Engine and they will start to be monitored
**End of February:
***Present at the OMB the results of the pilot and decision for the next step regarding the production infrastructure
* For site-admins: use the [https://gocdb-test.esc.rl.ac.uk/portal/index.php GOC-DB test instance] for testing the procedure
** Follow the [https://wiki.egi.eu/wiki/HOWTO21 How to add URL service endpoint information into GOC-DB]


== VAPOR ==
== VAPOR ==
Line 94: Line 120:
** we need your feedback to improve the service
** we need your feedback to improve the service
** some known issues will be fixed in the next release
** some known issues will be fixed in the next release
* new version 2.2 will be released this month:
* new version 2.2 is about to be released:
** please test it going on the dev instance http://operations-portal.egi.eu/vapor_dev
** please test it going on the dev instance http://operations-portal.egi.eu/vapor_dev
** report any comment into https://ggus.eu/index.php?mode=ticket_info&ticket_id=124872
** report any comment into https://ggus.eu/index.php?mode=ticket_info&ticket_id=124872

Latest revision as of 15:37, 5 December 2016


General information

UMD/CMD/Preview

  • UMD 4.3.0 'October' released http://repository.egi.eu/2016/11/10/release-umd-4-3-0/
    • ARC, GFAL2, XROOT, Davix, dCache, ARGUS, Gridsite, edg-mkgrid, umd-release for CentOS7
    • ARC, GFAL2, XROOT, Gridsite, edg-mkgrid, umd-release, GRAM5, DPM, Globus GridFTP, globus-default-security, MyProxy, Davix, dCache, VOMS, YAIM core, lcas-lcmaps for SL6
    • update: UMD 4.3.1 http://repository.egi.eu/2016/11/23/release-umd-4-3-1/
    • CentOS7: lcas-lcmaps-gt4-interface 0.3.0-0.3.1, lcmaps 1.6.6, lcas 1.3.19, glExec 1.2.3, glExec-WN 1.3.0, lcmaps-plugins 1.7.1
    • SL6: ARGUS server 1.7



  • CMD-ONE (OpenNebula 4 version) just started, will include products from Indigo as well

Preview repository

  • Released on 2016-11-25:
    • Preview 1.6.0 AppDB info (sl6): ARC 15.03 update 10, frontier-squid 3.5.22-2.1, gfal2 utility 1.4.0, XRootD 4.5.0
    • Preview 2.6.0 AppDB info (CentOS 7): ARC 15.03 update 10, dCache 2.16.18, frontier-squid 3.5.22-2.1, gfal2 utility 1.4.0, XRootD 4.5.0, UI/WN tarball.

Note: EGI provides the preview repository without any additional quality assurance process, but the products are released as they are provided by the product team. EGI recommends the use of the UMD repositories, which contain software verified through the quality assurance process of UMD.

Operations

Decommissioning of mon.egi.eu

  • tomorrow, December 6th 2016 ARGO team will decommission the old SAM GridMon box mon.egi.eu, housing central ATP and POEM. These services became obsolete when we switched to central monitoring instances in July.
    • VO SAM instances will not be affected as they are using local ATP and POEM.
    • Remaining NGI SAM instances rely on central ATP and will no longer get topology updates, so this gives you extra incentive to decommission them.

cloudmon.egi.eu dismissed

Software upgrades for OpenStack cloud RCs

  • keystone-VOMS and cloud-info-provider updates available, need to be installed on all OpenStack sites
  • as keystone-VOMS last version is only compatible with Liberty and Mitaka, in case OpenStack is Kilo (or older) an upgrade plan of OpenStack has been asked
  • according to EGI policies https://wiki.egi.eu/wiki/PROC16_Decommissioning_of_unsupported_software OpenStack Kilo or older should NOT be running on the infrastructure! we are asking for discussing this point at the next OMB (October 27)
  • as many sites are finding difficulties in planning upgrades against the very tight release cycle of OpenStack, please come with suggestion and reply with details in the tickets in order to shape the best (shared) proposal
  • ticket campaign ONGOING for all OpenStack sites, asked to upgrade to keystone-VOMS >=8.0.3, cloud-info-provider >=0.6, and plans for the future (OpenStack version currently deployed, plans for upgrades, usual specific RC upgrade schedule), UPDATE:
    • INDIGO-CATANIA-STACK and INFN-CATANIA-STACK moving to Mitaka (no plan)
    • IISAS-GPUCloud Liberty
    • FZJ user isolation bug fixed in Newton, not in Mitaka (investigating about a backport to Mitaka), waiting for solution
    • IN2P3-IRES Mitaka
    • CETA-GRID using Icehouse, planning mid-term upgrade (Newton?)
    • IISAS-FedCloud Mitaka from Ubuntu 16.04 LTS installed
    • BIFI upgrading to Mitaka
    • SCAI upgraded to Mitaka
    • INFN-PADOVA-STACK FIXED using Liberty
    • IFCA-LCG2 using Liberty
    • CYFRONET-CLOUD running Juno, evaluating Mitaka
    • TR-FC1-ULAKBIM FIXED using Liberty
    • NCG-INGRID-PT, using Mitaka, up to date
    • RECAS-BARI preparing upgrade to Mitaka from Ubuntu 16.04 LTS (deadline by end of year)
    • 100IT Liberty (evaluating Mitaka)

Monthly Availability/Reliability

  • November A/R figures not definitive yet
  • Underperformed sites in the past A/R reports with issues not yet fixed:
    • NGI_DE GGUS 123836
      • TUDresden-ZIH: set-up a new CREAM-CE, the CA probes were failing. Issues on SRM service
    • NGI_IBERGRID: GGUS 124371
      • CIEMATIC-TIC: decommissioning the storage element
    • NGI_NL: GGUS 123532
      • BelGrid-UCL: UNKNOWN status returned by CREAM probes, asked a recomputation
  • Sites suspended after October A/R report:
    • UNI-DORTMUNT (NGI_DE): downtime longer than a month
    • TW-eScience (AsiaPacific): long standing issues
    • INFN-CAGLIARI (NGI_IT): underperorming since too many months, never recovered

ARGO proposal to use GOCDB as the only source of topology information

  • Timescale:
    • New GOC-DB release on Dec 7th including a boolean ‘monitored’ flag for the service endpoints
    • Then creation of a web UI view for uncertified sites in ARGO
    • Uncertified sites will be asked to fill in the service endpoints information. Follow the How to add URL service endpoint information into GOC-DB
    • As information is added in the GOCDB, uncertified sites/services will be picked up by the ARGO Monitoring Engine and they will start to be monitored
    • By Q2 2017: support for multiple service endpoints

VAPOR

AOB

Next meeting

Retrieved from "https://wiki.egi.eu/w/index.php?title=Agenda-05-12-2016&oldid=91955"