Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI-Engage:TASK JRA1.4 Operations Tools

From EGIWiki
Jump to navigation Jump to search
EGI-Engage project: Main page WP1(NA1) WP3(JRA1) WP5(SA1) PMB Deliverables and Milestones Quality Plan Risk Plan Data Plan
Roles and
responsibilities
WP2(NA2) WP4(JRA2) WP6(SA2) AMB Software and services Metrics Project Office Procedures


Task leader: Cyril Lorphelin/CNRS
Task leader deputy: TBD

Mailing list:TBD


Involved Partner

  • CNRS
  • CESNET
  • GRNET
  • STFC
  • SRCE

Releases

Roadmap

Operations Portal

Task number

(4.1.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Planned/Done)

Dependencies from other tasks

(insert only task number)

Issues
4.1.1

LavUpgrade

Upgrade Vapor configuration for lavoisier

05/2015
02/2016
Released



4.1.2

LavGlue

Replace Glue1.3 queries by Glue 2

05/2015
02/2016
Released 4.1.1

4.1.3

LavCloud

Catpure Cloud Resources in Lavoisier

06/2015 02/2016 Released

4.1.4

RBExtension

Extend the resource browser to cloud resources

06/2015
02/2016 Released 4.1.3

4.1.5

VaporExtension

Extend Vapor views to cloud resources

06/2015 02/2016 Released 4.1.3

4.1.6

DashboardExtension

Extend dashboard views to cloud resources

10/2015
02/2016 Released 4.1.3

4.1.7

ReplaceGstat

Add Gstat Main Features into Glue2Browser

11/2015
02/2017
On-going


4.1.8

ExtendAPI1

Extend dashboard API

04/2016 02/2017 Planned

4.1.9

ExtendAPI2

Extend Vapor API

04/2016 02/2017 Released

4.1.10

ExtendAPI3

Extend Glue2Browser API

04/2016 02/2017 Released 4.1.7
4.1.11

MonitorVM1

monitor running/creation requests VM

06/2016
02/2017 Planned


4.1.12

MonitorVM2

monitor success/error/time-out rates for cloud sites

06/2016 02/2017 Planned
4.1.11

4.1.13

ExtendVoDataManagement

Support cloud storage solutions / File catalog

01/2017 08/2017 Planned

4.1.14 OpsPortalEgiMetrics
Extends the current metrics and add new ones
05/2016 09/2016 Released

4.1.15

OpsPortalVoAcknowledgement

Add Vo Acknowledgement section in VO ID card

10/2016 11/2016 Released

4.1.16

OpsPortalVoRobotCertificate

Add certificate robot section

11/2016 12/2016 Released

GOCDB

  • Note, targets and dates are indicative and subject to change and augmentation as requirements/priorities evolve.

Task number

(4.2.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Planned/Done)

Dependencies from other tasks

(insert only task number)

Issues
4.2.0 Extend data model 03/15 08/17 Done/ongoing
4.2.1 v5.4
  • RoleActionLogging
  • Fine grained content rendering (PermitAll and Protected pages)
  • Downtime declaration in local timezone
04/15 07/15 Done
4.2.2 v5.5 06/15 12/15 Done
  • Issues related to data protection and the Elixir/GEANT CoCo delay the production release of FIM. This is an EGI wide issue - all ops tools need to use the same eppn (most likely hashed given data protection concerns) in order to correlate accounts across services.
4.2.3 D3.4: First release of the Operational tools - GOCDB 02/16 02/16 Done
4.2.4 v5.6
  • Reserved Scope Tags - Restrict user tagging of resources using reserved/protected scope tags. Will allow WLCG/Elixir sites to apply tags only to their resources and prevent other sites from using same tags.
  • Downtime Calendar with fine grained filtering by (reserved) scopes and other params.
  • Bulk addition/upload of multiple custom properties including support for .properties file uploads (https://rt.egi.eu/rt/Ticket/Display.html?id=9427)
  • Integration with EGI ProxyIdp - Allow access for users without client certificate.
11/15 07/03/16 Done
4.2.4.1 v5.7
  • Optional Paging on PI - Selected PI queries should page by default. If page URL parameters are not provided, then query will return the first page of results by default. As a minimum, apply to get_downtime*, get_service* methods and review other methods for paging. Use cursor-based paging and not offset-based paging.
05/16 07/16 12/16 Done Announced for early/EGI testing 09/08/2016. Was delayed in order to implement new cursor-based paging logic which is required to solve data inconsistency issues associated with offset-based paging.

Released as part of v5.7 07/12/2016

4.2.5 v5.8 features merged into v5.7
  • Writable REST API to post updates to sites/services (requested by WLCG and EUDAT). Requires Site's to manage their own DN based ACLs per site required to post updates for a site/service.
  • Add new monitoring attributes for ARGO Enables EGI ARGO service to removed dependency on BDII
03/16 09/16 12/16 Done Planned alpha release W/C September 19th. Further details on dedicated wiki page. Planned alpha release w/c 24th Oct.

Released as part of v5.7 release 07/12/16

4.2.6
  • Schedule OTAG to re-prioritise GOCDB developments for next release Make request to EGI for GOCDB OTAG.
01/17 ~02/17 Completed (27/02/2017) Details: https://indico.egi.eu/indico/event/3296/
4.2.7 D3.11: Second release of the Operational tools - GOCDB 02/17 02/17 Completed 4.2.4, 4.2.4.1, 4.2.5, 4.2.6
4.2.8 Write API Extensions for Services/ServiceEndpoints 03/17 07/17 Planned
4.2.9 Verify Data Freshness Check 03/17 07/17 Planned
4.2.10 GUI Improvements 03/17 07/17 Planned No API changes - GUI only meaning the terminology in the API and GUI would diverge.
4.2.11 List of smaller issues reported in Github 03/17 07/17 Planned
4.2.12 ~Downtime classification changes 03/17 07/17 Tentative If DT > 1 month in duration, declare 1 month in advance to be SCHEDULED. If DT < than 5 days, can declare 24hrs in advance to be SCHEDULED. If there are no requirements for complex DT rules based on different Site/SE groupings allowing the same rules to apply to all DTs, then this could be implemented. Pending - To agree with WLCG/EGI
4.2.13 ~Unique Constraint on hostName/serviceType pair 03/17 07/17 Tentative Not an EGI requirement, but happy to re-prioritise if still needed by WLCG.
4.2.14 ~NGI Certification Status Rules 03/17 07/17 Tentative Useful to have a quick way of changing CertStatus of all Sites belonging to an NGI. Determine if following account portal requirement is related/needed: https://rt.egi.eu/rt/Ticket/Display.html?id=11481
4.2.15 D3.18: Final release of the Operational tools - GOCDB 08/17 08/17 Planned 4.2.8 to 4.2.14
E-Infra12 proposals
  • Replace UI with modern Web framework
  • More Change logging
  • New requirements will probably emerge relating to dynamic attributes and extending GOCDB in info-service space
  • DB migration scripts for higher resilience

Monitoring

Task number

(4.3.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Planned/Done)

Dependencies from other tasks

(insert only task number)

Issues
4.3.1 ARGO Compute Engine & Web API:
  • automatic recomputation triggers
  • multi-tenant support
  • stability and performance improvements
04/15 06/15 Planned
4.3.2 ARGO Monitoring Engine:
  • probe framework
  • support documentation (Guides)
  • stability and performance improvements
04/15 06/15 Planned
4.3.3 ARGO EGI Web UI
  • ACL mechanism (support groups/roles)
  • UI Enhancements
04/15 06/15 Planned
4.3.4 ARGO EGI Connectors & Consumer:
  • improved support for VOs
  • stability and performance improvements
04/15 06/15 Planned
4.3.5 ARGO POEM:
  • ACL mechanism (support groups/roles)
  • stability and performance improvements
04/15 06/15 Planned
4.3.6 ARGO Compute Engine & Web API:
  • API for data ingestion specification
  • separation of A/R and Metric stores
  • APIv2 Specification
  • stability and performance improvements
07/15 09/15 Planned 4.3.1
4.3.7 ARGO Monitoring Engine:
  • fedcloud probes
  • stability and performance improvements
07/15 09/15 Planned 4.3.2
4.3.8 ARGO EGI Web UI:
  • UI Enhancements
  • initial support for federated logins using SAML
07/15 09/15 Planned 4.3.3
4.3.9 ARGO EGI Connectors & Consumer:
  • use of CE ingestion api
  • stability and performance improvements
07/15 09/15 Planned 4.3.4
4.3.10 ARGO POEM
  • initial support for federated logins using SAML
  • support for probe management
  • stability and performance improvements
07/15 09/15 Planned 4.3.5
4.3.11 ARGO Compute Engine & Web API:
  • API for data ingestion specification
  • APIv2
  • stability and performance improvements
09/15 12/15 Planned 4.3.6
4.3.12 ARGO Monitoring Engine:
  • fedcloud probes update
  • stability and performance improvements
09/15 12/15 Planned 4.3.7
4.3.13 ARGO EGI Web UI:
  • UI Enhancements
  • support for federated logins with IdP Discovery
09/15 12/15 Planned 4.3.8
4.3.14 ARGO EGI Connectors & Consumer:
  • stability and performance improvements
09/15 12/15 Planned 4.3.9
4.3.15 ARGO POEM:
  • support for federated logins with IdP Discovery
  • stability and performance improvements
09/15 12/15 Planned 4.3.10
4.3.16 ARGO Compute Engine & Web API:
  • stability and performance improvements
01/16 03/16 Planned 4.3.11
4.3.17 ARGO Monitoring Engine:
  • fedcloud probes update
  • stability and performance improvements
01/16 03/16 Planned 4.3.12
4.3.18 ARGO EGI Web UI:
  • join as a SP to eduGAIN
01/16 03/16 Planned 4.3.13
4.3.19 ARGO EGI Connectors & Consumer
  • stability and performance improvements
01/16 03/16 Planned 4.3.14
4.3.20 ARGO POEM
  • join as a SP to eduGAIN
  • stability and performance improvements
01/16 03/16 Planned 4.3.15

Messaging

Task number

(4.4.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Planned/Done)

Dependencies from other tasks

(insert only task number)

Issues
4.4.1 Preparatory phase 04/05 06/05 Planned
4.4.2 APIv1 alpha specification 07/05 09/05 Planned 4.4.1
4.4.3 APIv1 test implementation

APIv1 final draft specification (ready for external party review)

09/05 12/05 Planned 4.4.2
4.4.4 APIv1 final specification 01/06 03/06 Planned 4.4.2

Security Monitoring

Task number

(4.5.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Planned/Done)

Dependencies from other tasks

(insert only task number)

Issues