Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "GOCDB/Release4/Development"

From EGIWiki
Jump to navigation Jump to search
(Replaced content with "{{Template:Op menubar}} {{Template:GOCDB_menubar}} This page has moved to GOCDB/Roadmap")
 
(149 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Back to [[GOCDB/Documentation_Index]] <br/>
{{Template:Op menubar}}
{{Template:GOCDB_menubar}}


==Recent Completed Developments==
This page has moved to [[GOCDB/Roadmap]]
 
* New Downtime interface much improved, moved to single page + can now select all endpoints under one site - September 2011
* Refactored database code to be atomic (required to implement scoping) - July 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=943
* Moved Site, User and Endpoint manipulation to new MVC architecture ready to support scoping - July 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=943
* Fixed a few long term bugs with adding and editing sites and users - July 2011
* Increased front end responsiveness - July 2011
* Cleaned old roles - June 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=931
* Added a URL field to service endpoint objects to support UNICORE services - May 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=975 (note, even though a single URL field is displayed per Endpoint, a new Gocdb_Endpoint_Location entity was created in order to link multiple service endpoint locations per SE if this becomes a future requirement - see [[Media:GocdbGlue2UnicoreV2.pdf]]).
* Add a "my site" link in the main menu - May 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=1091 (note, this uses the new controller-per page with templates design as summarized below).
* Record Certification Status History - May 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=939
* Allow authorised operators to suspend sites at central level - March 2011 https://rt.egi.eu/rt/Ticket/Display.html?id=1094
* Show when a site entered the current production status - March 2011 https://wiki.egi.eu/wiki/GOCDB/PI/get_cert_status_date
 
== Current Developments ==
Regionalisation related developments are given in their own page: [[GOCDB/Release4/Regionalisation]]<br />
Feedback received on the new system is given at: [[GOCDB/Release4/Feedback]]
 
The list of [[https://rt.egi.eu/rt/Search/Results.html?Format=%27%20%20%20%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__id__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3A%23%27%2C%0A%27%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__Subject__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3ASubject%27%2C%0A%27__Status__%27%2C%0A%27__QueueName__%27%2C%0A%27__OwnerName__%27%2C%0A%27__Priority__%27%2C%0A%27__NEWLINE__%27%2C%0A%27%27%2C%0A%27%3Csmall%3E__Requestors__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__CreatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__ToldRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__LastUpdatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__TimeLeft__%3C%2Fsmall%3E%27&Order=ASC|ASC|ASC|ASC&OrderBy=id|||&Query=Queue%20%3D%20%27requirements%27%20AND%20%28%20%20Status%20%3D%20%27new%27%20OR%20Status%20%3D%20%27open%27%20OR%20Status%20%3D%20%27accepted%27%20OR%20Status%20%3D%20%27developed%27%20OR%20Status%20%3D%20%27stalled%27%20OR%20Status%20%3D%20%27feedback%27%20%29%20AND%20%27CF.{Category%20%28level%202%29}%27%20LIKE%20%27GOCDB%20%28Operational%20Tools%29%27%20AND%20Queue%20%3D%20%27requirements%27%20AND%20%27CF.{Category%20%28level%201%29}%27%20LIKE%20%27Operational%20Tools%27&RowsPerPage=50&SavedChartSearchId=new&SavedSearchId=RT%3A%3AUser-3851-SavedSearch-1174 GOCDB development items is available in the EGI RT ticket tracker]]
 
These are the developments we are currently working on. They come from the agreed development list, as defined by the Operational Tools Advisory Group (OTAG) who filters and prioritizes user requests.
 
=== Data Scoping in the Central GOCDB ===
The following tasks need to be completed to introduce scoping to the central GOCDB instance. "DONE" means completed in our current development version of GOCDB and not necessarily put into production.
 
* Enable database level tagging to allow scoping of data - <b>DONE</b>
* Modify PI queries to be scope aware (e.g. get_downtime?scope=Regional) - <b>DONE</b>
* Atomic database access - <b>IN PROGRESS</b>
* Add scoping capability to the web portal - <b>IN PROGRESS</b>
** This is being achieved through MVC implementation of inserting / editing objects
*** Add / Edit Service Endpoint - <b>DONE</b>
*** Add / Edit Site - <b>DONE</b>
*** Add / Edit Downtime  - <b>DONE</b>
*** Add / Edit User - <b>DONE</b>
*** Add / Edit Role - <b>IN PROGRESS</b>
* Remove old Non-MVC code - <b>IN PROGRESS</b>
* Remove old XML Input code
* Indicate what scope data has in the web portal
** When viewing an individual object
** When viewing a table of objects
* Add scope specific web portal views
** View Local Sites
** View EGI Sites
** Scope Specific Search
 
=== Development Notes and Other Developments ===
* <b>The GUI logic is being re-developed.</b> Currently GOCDB4 uses a generic module for drawing a forms and GUI components such as tables. This proprietary module uses a single code path for all form and table based operations. In doing this, the code has become too rigid making it difficult to deal with form/table requirements on a per-page basis. This module will be replaced with a more flexible controller per page architecture with view templates. This is necessary to address a number of the RT requirement requests (approximately half suggest GUI improvements).
 
* <b>The object cardinality logic needs to be refactored</b> to fix known issues, particularly Domain, Production-Status and Service-Endpoint objects.
 
* <b>Atomic database transactions and fix tx demarcation issues</b> (e.g. on site creation Savannah 74860). Currently, the GOCDB PROM API does not allow demarcation of transaction boundaries within high level business service layers (e.g. as in a regular stateless service facade). At present, explicit commits and rollbacks are declared in low-level PLSQL functions including fnNewLink(), fnUpdateObject(), fnDeleteObject(), fnDeleteLink(). Consequently, any business operation that invokes multiple low level operations such as inserting a site and adding new SEs cannot declare tx boundaries around the whole process. These operations are therefore not atomic and can leave the db in an inconsistent state if an error occurs (since rollback over all the low level operations is not possible and probably accounts for some inconsistencies seen in the db). The PROM PL/SQL layer and the higher level DAO and Service layers require refactoring so that transaction demarcation can be moved into the business operations.
 
* <b>Explicitly indicate if all SEs of a site are in downtime through the PI</b>
 
* <b>Redevelop the xml_output module to implement nested XML collections,</b> probably using the Query2XML package. This is necessary because the existing XML Output module will only generate flat XML documents (e.g. that often map to individual DB entities). Currently SQL joins/associations between Sites, SEs and URLs cannot be represented as hierarchical/nested XML documents (for example, consider nesting multiple EndpointLocation objects/URLs in the ServiceEndpoint XML, see: [[Media:GocdbGlue2UnicoreV2.pdf]]).
 
* <b>Support for 'Virtual Site Entities (VSite)'</b> There is a requirement to group existing Service Endpoints (currently grouped under their corresponding 'owning' physical Site) under a new 'Virtual Site' entity. See: https://rt.egi.eu/rt/Ticket/Display.html?id=987 and https://wiki.egi.eu/wiki/Logical-site
** At present, the proposed features of a VSite will include (see [[Media:VirtualSitesDesign.pdf‎]]):
*** A VSite will be a separate GOCDB entity, and will have users and other attributes much like an existing physical site but with different rules (described below). Importantly, we believe this grouping has to be a new gocdb object in order to clearly define the intended semantics of the grouping (e.g. both a 'VSite' and an imaginary 'VService' could both group many SEs, but a VSite has very different semantics to a VService !). 
*** A VSite will be used to group existing Service Endpoints only (i.e. SEs that have already been created under their owning physical Site).
*** A VSite cannot group new SEs that have no owning physical site.
*** A single SE may only have a single parent physical Site (i.e. GOCDB cardinality of 1 between Site and SE)
*** A single SE can have many parent VSites (requires GOCDB cardinality of 'many-to-many' between Virtual Site and SE).
*** New PI queries are proposed to support querying of VSites and querying of SEs that are grouped under a VSite (much like the existing get_site and get_service_endpoint methods (https://wiki.egi.eu/wiki/GOCDB/PI/get_service_endpoint_method and https://wiki.egi.eu/wiki/GOCDB/PI/get_site_method ). The following XML example is related: https://twiki.cern.ch/twiki/pub/Main/ATPVOFeeds/atp_vo_feed_example.xml 
*** The permissions model of a VSite is not yet well defined. It is currently proposed that users with a role over the owning physical site should maintain their cascading permissions over their SEs (i.e. no modification to the current site/permissions model), however, users with a role over the VSite will not have any cascading permissions over the SEs. Consequently, this has the following important implications;
**** A VSite could not be used to declare a downtime for all its member SEs.
**** Similarly, users with a role over the VS will not be able to update/modify a member SE.
*** If VSite permissions are required (e.g. for declaring SE downtimes and modifying SEs), then a user may have to request a new role under the physical site or be granted a corresponding permission. These requirements are currently undefined.
 
* <b>The GOCDB role/permissions model may need to be updated</b> in order to cater for a finer grained permission proposal currently emerging as a new EGI requirement. [[Media:FinerGrainedGOCDB_rolesVeraProposal2.xls]]  
 
 
{|{{egi-table}}
! development item !! Estimated release !! Estimated date
|-
| Provide a production quality package for GOCDB regional module || 4.0d7 || Oct 2010
|-
| Improvement of GOCDB failover system and backend replication || 4.0.1 || May 2011
|-
| New service endpoint URL associations for new service types (see above) || ? || April/June 2011
|- 
| Re-develop GUI Logic (see above) || ? || Sept 2011 (EGI-Y2)
|-
| Update/refactor role model for finer grained permissions (see above) || ? || EGI-Y2
|-
| Data scoping by tagging GOCDB entities (see above) || ? || EGI-Y2
|-
| Regionalisation [[GOCDB/Release4/Regionalisation]] || ? || EGI-Y2/3
|-
| Work on GOCDB and Operations Portal common front end || ? || EGI-Y2
|-
| GOCDB interface to the dynamic information system (BDII/GluE) || ? || EGI-Y3
|}
 
 
<!--
| Consolidation and polishing of the web admin interface for GOCDB regional module || 4.0.2 || Mar 2011
|-
| Provide a SOAP interface to GOCDB in parallel to GOCDBPI || 4.0.2 || Mar 2011
|-
| Provide a MySQL version of the GOCDB regional module || 4.1 || May 2011
|-
| GOCDB interface to MyEGEE framework or equivalent || ? || EGI-Y2
|-
-->

Latest revision as of 09:04, 2 April 2015