Difference between revisions of "EGI-Engage:TASK JRA1.4 Operations Tools"
(→GOCDB) |
m (→GOCDB) |
||
(44 intermediate revisions by 4 users not shown) | |||
Line 58: | Line 58: | ||
| 05/2015<br> | | 05/2015<br> | ||
| 02/2016<br> | | 02/2016<br> | ||
| | | Released | ||
| <br> | | <br> | ||
| | | | ||
Line 74: | Line 74: | ||
| 05/2015<br> | | 05/2015<br> | ||
| 02/2016<br> | | 02/2016<br> | ||
| | | Released | ||
| 4.1.1<br> | | 4.1.1<br> | ||
| <br> | | <br> | ||
Line 86: | Line 86: | ||
| 06/2015 | | 06/2015 | ||
| 02/2016 | | 02/2016 | ||
| | | Released | ||
| | | <br> | ||
| | | <br> | ||
|- | |- | ||
! 4.1.4<br> | ! 4.1.4<br> | ||
Line 98: | Line 98: | ||
| 06/2015<br> | | 06/2015<br> | ||
| 02/2016 | | 02/2016 | ||
| | | Released | ||
| 4.1.3<br> | | 4.1.3<br> | ||
| <br> | | <br> | ||
Line 110: | Line 110: | ||
| 06/2015 | | 06/2015 | ||
| 02/2016 | | 02/2016 | ||
| | | Released | ||
| 4.1.3<br> | | 4.1.3<br> | ||
| <br> | | <br> | ||
Line 121: | Line 121: | ||
| 10/2015<br> | | 10/2015<br> | ||
| 02/2016 | | 02/2016 | ||
| | | Released | ||
| 4.1.3<br> | | 4.1.3<br> | ||
| <br> | | <br> | ||
Line 130: | Line 130: | ||
ReplaceGstat | ReplaceGstat | ||
Add Gstat Main Features into Glue2Browser<br> | |||
| 11/2015<br> | | 11/2015<br> | ||
| 02/2017<br> | | 02/2017<br> | ||
| | | On-going<br> | ||
| <br> | | <br> | ||
| <br> | | <br> | ||
Line 145: | Line 145: | ||
| 04/2016 | | 04/2016 | ||
| 02/2017 | | 02/2017 | ||
| Planned | | Planned | ||
| | | <br> | ||
| | | <br> | ||
|- | |- | ||
! 4.1.9 | ! 4.1.9 | ||
Line 158: | Line 158: | ||
| 04/2016 | | 04/2016 | ||
| 02/2017 | | 02/2017 | ||
| | | Released | ||
| | | <br> | ||
| | | <br> | ||
|- | |- | ||
! 4.1.10 | ! 4.1.10 | ||
Line 166: | Line 166: | ||
ExtendAPI3 | ExtendAPI3 | ||
Extend | Extend Glue2Browser API | ||
| 04/2016 | | 04/2016 | ||
| 02/2017 | | 02/2017 | ||
| | | Released | ||
| 4.1.7 | | 4.1.7 | ||
| | | <br> | ||
|- | |- | ||
! 4.1.11<br> | ! 4.1.11<br> | ||
Line 181: | Line 181: | ||
| 06/2016<br> | | 06/2016<br> | ||
| 02/2017 | | 02/2017 | ||
| Planned<br> | | Planned<br> | ||
| <br> | | <br> | ||
Line 207: | Line 207: | ||
| 08/2017 | | 08/2017 | ||
| Planned | | Planned | ||
| <br> | |||
| <br> | |||
|- | |||
! 4.1.14 | |||
| OpsPortalEgiMetrics<br>Extends the current metrics and add new ones | |||
| 05/2016 | |||
| 09/2016 | |||
| Released | |||
| <br> | |||
| <br> | |||
|- | |||
! 4.1.15 | |||
| | | | ||
OpsPortalVoAcknowledgement | |||
Add Vo Acknowledgement section in VO ID card | |||
| 10/2016 | |||
| 11/2016 | |||
| Released | |||
| <br> | |||
| <br> | |||
|- | |||
! 4.1.16 | |||
| | | | ||
OpsPortalVoRobotCertificate | |||
Add certificate robot section | |||
| 11/2016 | |||
| 12/2016 | |||
| Released | |||
| <br> | |||
| <br> | |||
|} | |} | ||
== GOCDB == | == GOCDB == | ||
* Note, targets and dates are indicative and subject to change and augmentation as requirements/priorities evolve. | |||
*Note, targets and dates are indicative and subject to change and augmentation as requirements/priorities evolve. | |||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
Line 218: | Line 252: | ||
Task number | Task number | ||
(4.2.X) | (4.2.X) | ||
! Task name | ! Task name | ||
Line 224: | Line 258: | ||
Start date | Start date | ||
(MM/YY) | (MM/YY) | ||
! | ! | ||
Release Date | Release Date | ||
(MM/YY) | (MM/YY) | ||
! | ! | ||
Line 241: | Line 275: | ||
(insert only task number) | (insert only task number) | ||
! Issues | ! Issues | ||
|- | |||
| 4.2.0 | |||
| Extend data model | |||
| 03/15 | |||
| 08/17 | |||
| Done/ongoing | |||
| | |||
|- | |- | ||
| | | 4.2.1 | ||
| | | v5.4 | ||
* RoleActionLogging | *'''RoleActionLogging''' | ||
* | *'''Fine grained content rendering '''(PermitAll and Protected pages) | ||
* Downtime declaration in local timezone | *'''Downtime declaration in local timezone''' | ||
| | |||
| | | 04/15 | ||
| | | 07/15 | ||
| | | Done | ||
| | | | ||
| | |||
|- | |||
| 4.2.2 | |||
| v5.5 | |||
*'''Role Abstractions for Multi-tenant''' (host multiple projects in same goc instance and apply different roles per-project - users can have roles in different projects https://rt.egi.eu/rt/Ticket/Display.html?id=9429) | |||
*'''SAML/FIM authentication''', Elixir (https://rt.egi.eu/rt/Ticket/Display.html?id=7493) | |||
| 06/15 | |||
| 12/15 | |||
| Done | |||
| | |||
| | |||
*Issues related to data protection and the Elixir/GEANT CoCo delay the production release of FIM. This is an EGI wide issue - all ops tools need to use the same eppn (most likely hashed given data protection concerns) in order to correlate accounts across services. | |||
|- | |||
| 4.2.3 | |||
| D3.4: First release of the Operational tools - GOCDB | |||
| 02/16 | |||
| 02/16 | |||
| Done | |||
| | |||
|- | |||
| 4.2.4 | |||
| v5.6 | |||
*'''Reserved Scope Tags''' - Restrict user tagging of resources using reserved/protected scope tags. Will allow WLCG/Elixir sites to apply tags only to their resources and prevent other sites from using same tags. | |||
*'''Downtime Calendar '''with fine grained filtering by (reserved) scopes and other params. | |||
*'''Bulk addition/upload of multiple custom properties''' including support for .properties file uploads (https://rt.egi.eu/rt/Ticket/Display.html?id=9427) | |||
* '''Integration with EGI ProxyIdp''' - Allow access for users without client certificate. | |||
| 11/15 | |||
| 07/03/16 | |||
| Done | |||
| | |||
| | |||
|- | |||
| 4.2.4.1 | |||
| v5.7 | |||
*'''Optional Paging on PI''' - Selected PI queries should page by default. If page URL parameters are not provided, then query will return the first page of results by default. As a minimum, apply to get_downtime*, get_service* methods and review other methods for paging. Use cursor-based paging and not offset-based paging. | |||
| 05/16 | |||
| <strike>07/16</strike> 12/16 | |||
| Done | |||
| | |||
| Announced for early/EGI testing 09/08/2016. Was delayed in order to implement new cursor-based paging logic which is required to solve data inconsistency issues associated with offset-based paging. | |||
Released as part of v5.7 07/12/2016 | |||
|- | |||
| 4.2.5 | |||
| <strike>v5.8</strike> features merged into v5.7 | |||
*'''Writable REST API '''to post updates to sites/services (requested by WLCG and EUDAT). Requires Site's to manage their own DN based ACLs per site required to post updates for a site/service. | |||
*'''Add new monitoring attributes for ARGO''' Enables EGI ARGO service to removed dependency on BDII | |||
| 03/16 | |||
| <strike>09/16</strike> 12/16 | |||
| Done | |||
| | |||
| <strike>Planned alpha release W/C September 19th.</strike> Further details on [[GOCDB/Write_API_Development|dedicated wiki page]]. Planned alpha release w/c 24th Oct. | |||
Released as part of v5.7 release 07/12/16 | |||
|- | |- | ||
| | | 4.2.6 | ||
| | | | ||
* | *'''Schedule OTAG to re-prioritise GOCDB developments for next release''' Make request to EGI for GOCDB OTAG. | ||
| 01/17 | |||
| | | ~02/17 | ||
| | | Completed (27/02/2017) | ||
| | | | ||
| Details: https://indico.egi.eu/indico/event/3296/ | |||
* | |- | ||
| 4.2.7 | |||
| '''D3.11: Second release of the Operational tools - GOCDB''' | |||
| 02/17 | |||
| 02/17 | |||
| Completed | |||
| 4.2.4, 4.2.4.1, 4.2.5, 4.2.6 | |||
| | |||
|- | |||
| 4.2.8 | |||
| '''Write API Extensions for Services/ServiceEndpoints''' | |||
* https://rt.egi.eu/rt/Ticket/Display.html?id=11020 | |||
| 03/17 | |||
| 07/17 | |||
| Planned | |||
| | |||
| | |||
|- | |- | ||
| | | 4.2.9 | ||
| | | '''Verify Data Freshness Check''' | ||
* https://rt.egi.eu/rt/Ticket/Display.html?id=8240 | |||
* | | 03/17 | ||
| 07/17 | |||
| | | Planned | ||
| | |||
| | |||
| | | | ||
| | |||
|- | |- | ||
| | | 4.2.10 | ||
| | | '''GUI Improvements''' | ||
| | * https://rt.egi.eu/rt/Ticket/Display.html?id=11546 | ||
| | * https://rt.egi.eu/rt/Ticket/Display.html?id=11217 | ||
| | | 03/17 | ||
| | | 07/17 | ||
| | | Planned | ||
| | |||
| No API changes - GUI only meaning the terminology in the API and GUI would diverge. | |||
|- | |- | ||
| | | 4.2.11 | ||
| | | '''List of smaller issues reported in Github''' | ||
| | * https://github.com/GOCDB/gocdb/issues | ||
| | | 03/17 | ||
| | | 07/17 | ||
| | | Planned | ||
| | |||
| | |||
|- | |||
| 4.2.12 | |||
| '''~Downtime classification changes''' | |||
* Tentative - depends on complexity of requirements | |||
* https://rt.egi.eu/rt/Ticket/Display.html?id=10845 | |||
| 03/17 | |||
| 07/17 | |||
| Tentative | |||
| | |||
| If DT > 1 month in duration, declare 1 month in advance to be SCHEDULED. If DT < than 5 days, can declare 24hrs in advance to be SCHEDULED. If there are no requirements for complex DT rules based on different Site/SE groupings allowing the same rules to apply to all DTs, then this could be implemented. Pending - To agree with WLCG/EGI | |||
|- | |- | ||
| | | 4.2.13 | ||
| | | '''~Unique Constraint on hostName/serviceType pair''' | ||
* | * Tentative - may not be a requirement any longer | ||
* | * https://rt.egi.eu/rt/Ticket/Display.html?id=10368 | ||
| | | 03/17 | ||
| | | 07/17 | ||
| | | Tentative | ||
| | | | ||
| | | Not an EGI requirement, but happy to re-prioritise if still needed by WLCG. | ||
|- | |- | ||
| | | 4.2.14 | ||
| | | '''~NGI Certification Status Rules''' | ||
* | * Tentative - Nice to have, depends on remaining effort | ||
| | * https://rt.egi.eu/rt/Ticket/Display.html?id=9084 | ||
| | | 03/17 | ||
| | | 07/17 | ||
| | | Tentative | ||
| | | | ||
| Useful to have a quick way of changing CertStatus of all Sites belonging to an NGI. Determine if following account portal requirement is related/needed: https://rt.egi.eu/rt/Ticket/Display.html?id=11481 | |||
|- | |- | ||
| | | 4.2.15 | ||
| | | '''D3.18: Final release of the Operational tools - GOCDB''' | ||
| | | 08/17 | ||
| | | 08/17 | ||
| | | Planned | ||
| | | 4.2.8 to 4.2.14 | ||
| | | | ||
|- | |- | ||
| | | | ||
| | | '''E-Infra12 proposals''' | ||
| | * Replace UI with modern Web framework | ||
| | * More Change logging | ||
| | * New requirements will probably emerge relating to dynamic attributes and extending GOCDB in info-service space | ||
| | * DB migration scripts for higher resilience | ||
| | | | ||
| | |||
| | |||
| | |||
| | |||
|} | |} | ||
Line 356: | Line 480: | ||
Status | Status | ||
( | (Planned/Done) | ||
! | ! | ||
Line 591: | Line 715: | ||
Status | Status | ||
( | (Planned/Done) | ||
! | ! | ||
Line 657: | Line 781: | ||
Status | Status | ||
( | (Planned/Done) | ||
! | ! |
Latest revision as of 15:33, 24 March 2017
Task leader: Cyril Lorphelin/CNRS
Task leader deputy: TBD
Mailing list:TBD
Involved Partner
- CNRS
- CESNET
- GRNET
- STFC
- SRCE
Releases
Roadmap
Operations Portal
Task number (4.1.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Planned/Done) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|
4.1.1 |
LavUpgrade Upgrade Vapor configuration for lavoisier |
05/2015 |
02/2016 |
Released |
| |
4.1.2 |
LavGlue Replace Glue1.3 queries by Glue 2 |
05/2015 |
02/2016 |
Released | 4.1.1 |
|
4.1.3 |
LavCloud Catpure Cloud Resources in Lavoisier |
06/2015 | 02/2016 | Released | ||
4.1.4 |
RBExtension Extend the resource browser to cloud resources |
06/2015 |
02/2016 | Released | 4.1.3 |
|
4.1.5 |
VaporExtension Extend Vapor views to cloud resources |
06/2015 | 02/2016 | Released | 4.1.3 |
|
4.1.6 |
DashboardExtension Extend dashboard views to cloud resources |
10/2015 |
02/2016 | Released | 4.1.3 |
|
4.1.7 |
ReplaceGstat Add Gstat Main Features into Glue2Browser |
11/2015 |
02/2017 |
On-going |
||
4.1.8 |
ExtendAPI1 Extend dashboard API |
04/2016 | 02/2017 | Planned | ||
4.1.9 |
ExtendAPI2 Extend Vapor API |
04/2016 | 02/2017 | Released | ||
4.1.10 |
ExtendAPI3 Extend Glue2Browser API |
04/2016 | 02/2017 | Released | 4.1.7 | |
4.1.11 |
MonitorVM1 monitor running/creation requests VM |
06/2016 |
02/2017 | Planned |
||
4.1.12 |
MonitorVM2 monitor success/error/time-out rates for cloud sites |
06/2016 | 02/2017 | Planned |
4.1.11 |
|
4.1.13 |
ExtendVoDataManagement Support cloud storage solutions / File catalog |
01/2017 | 08/2017 | Planned | ||
4.1.14 | OpsPortalEgiMetrics Extends the current metrics and add new ones |
05/2016 | 09/2016 | Released | ||
4.1.15 |
OpsPortalVoAcknowledgement Add Vo Acknowledgement section in VO ID card |
10/2016 | 11/2016 | Released | ||
4.1.16 |
OpsPortalVoRobotCertificate Add certificate robot section |
11/2016 | 12/2016 | Released |
GOCDB
- Note, targets and dates are indicative and subject to change and augmentation as requirements/priorities evolve.
Task number (4.2.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Planned/Done) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|
4.2.0 | Extend data model | 03/15 | 08/17 | Done/ongoing | ||
4.2.1 | v5.4
|
04/15 | 07/15 | Done | ||
4.2.2 | v5.5
|
06/15 | 12/15 | Done |
| |
4.2.3 | D3.4: First release of the Operational tools - GOCDB | 02/16 | 02/16 | Done | ||
4.2.4 | v5.6
|
11/15 | 07/03/16 | Done | ||
4.2.4.1 | v5.7
|
05/16 | Done | Announced for early/EGI testing 09/08/2016. Was delayed in order to implement new cursor-based paging logic which is required to solve data inconsistency issues associated with offset-based paging.
Released as part of v5.7 07/12/2016 | ||
4.2.5 |
|
03/16 | Done | Released as part of v5.7 release 07/12/16 | ||
4.2.6 |
|
01/17 | ~02/17 | Completed (27/02/2017) | Details: https://indico.egi.eu/indico/event/3296/ | |
4.2.7 | D3.11: Second release of the Operational tools - GOCDB | 02/17 | 02/17 | Completed | 4.2.4, 4.2.4.1, 4.2.5, 4.2.6 | |
4.2.8 | Write API Extensions for Services/ServiceEndpoints | 03/17 | 07/17 | Planned | ||
4.2.9 | Verify Data Freshness Check | 03/17 | 07/17 | Planned | ||
4.2.10 | GUI Improvements | 03/17 | 07/17 | Planned | No API changes - GUI only meaning the terminology in the API and GUI would diverge. | |
4.2.11 | List of smaller issues reported in Github | 03/17 | 07/17 | Planned | ||
4.2.12 | ~Downtime classification changes
|
03/17 | 07/17 | Tentative | If DT > 1 month in duration, declare 1 month in advance to be SCHEDULED. If DT < than 5 days, can declare 24hrs in advance to be SCHEDULED. If there are no requirements for complex DT rules based on different Site/SE groupings allowing the same rules to apply to all DTs, then this could be implemented. Pending - To agree with WLCG/EGI | |
4.2.13 | ~Unique Constraint on hostName/serviceType pair
|
03/17 | 07/17 | Tentative | Not an EGI requirement, but happy to re-prioritise if still needed by WLCG. | |
4.2.14 | ~NGI Certification Status Rules
|
03/17 | 07/17 | Tentative | Useful to have a quick way of changing CertStatus of all Sites belonging to an NGI. Determine if following account portal requirement is related/needed: https://rt.egi.eu/rt/Ticket/Display.html?id=11481 | |
4.2.15 | D3.18: Final release of the Operational tools - GOCDB | 08/17 | 08/17 | Planned | 4.2.8 to 4.2.14 | |
E-Infra12 proposals
|
Monitoring
Task number (4.3.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Planned/Done) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|
4.3.1 | ARGO Compute Engine & Web API:
|
04/15 | 06/15 | Planned | ||
4.3.2 | ARGO Monitoring Engine:
|
04/15 | 06/15 | Planned | ||
4.3.3 | ARGO EGI Web UI
|
04/15 | 06/15 | Planned | ||
4.3.4 | ARGO EGI Connectors & Consumer:
|
04/15 | 06/15 | Planned | ||
4.3.5 | ARGO POEM:
|
04/15 | 06/15 | Planned | ||
4.3.6 | ARGO Compute Engine & Web API:
|
07/15 | 09/15 | Planned | 4.3.1 | |
4.3.7 | ARGO Monitoring Engine:
|
07/15 | 09/15 | Planned | 4.3.2 | |
4.3.8 | ARGO EGI Web UI:
|
07/15 | 09/15 | Planned | 4.3.3 | |
4.3.9 | ARGO EGI Connectors & Consumer:
|
07/15 | 09/15 | Planned | 4.3.4 | |
4.3.10 | ARGO POEM
|
07/15 | 09/15 | Planned | 4.3.5 | |
4.3.11 | ARGO Compute Engine & Web API:
|
09/15 | 12/15 | Planned | 4.3.6 | |
4.3.12 | ARGO Monitoring Engine:
|
09/15 | 12/15 | Planned | 4.3.7 | |
4.3.13 | ARGO EGI Web UI:
|
09/15 | 12/15 | Planned | 4.3.8 | |
4.3.14 | ARGO EGI Connectors & Consumer:
|
09/15 | 12/15 | Planned | 4.3.9 | |
4.3.15 | ARGO POEM:
|
09/15 | 12/15 | Planned | 4.3.10 | |
4.3.16 | ARGO Compute Engine & Web API:
|
01/16 | 03/16 | Planned | 4.3.11 | |
4.3.17 | ARGO Monitoring Engine:
|
01/16 | 03/16 | Planned | 4.3.12 | |
4.3.18 | ARGO EGI Web UI:
|
01/16 | 03/16 | Planned | 4.3.13 | |
4.3.19 | ARGO EGI Connectors & Consumer
|
01/16 | 03/16 | Planned | 4.3.14 | |
4.3.20 | ARGO POEM
|
01/16 | 03/16 | Planned | 4.3.15 |
Messaging
Task number (4.4.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Planned/Done) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|
4.4.1 | Preparatory phase | 04/05 | 06/05 | Planned | ||
4.4.2 | APIv1 alpha specification | 07/05 | 09/05 | Planned | 4.4.1 | |
4.4.3 | APIv1 test implementation
APIv1 final draft specification (ready for external party review) |
09/05 | 12/05 | Planned | 4.4.2 | |
4.4.4 | APIv1 final specification | 01/06 | 03/06 | Planned | 4.4.2 |
Security Monitoring
Task number (4.5.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Planned/Done) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|