GOCDB/Release4/Development/v5

From EGIWiki
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


<< Back to GOCDB/Release4/Development

v5 Release Summary

v5 Benefits of changing the DB model (dropping PROM for doctrine)

  • Write code once, deploy to Oracle, MySQL, Postgres, SQLite.
  • Much simpler than the GOCDB/PROM model (see query comparison below)]
  • Easier for other projects to extend / modify
  • Easier for new developers to learn
  • Better performance
  • Doctrine is de-facto object-relational-mapping library (ORM) for PHP.
    • Reduced risk of fail over performance issues
PROM SQL
Doctrine SQL

Drawbacks

  • Data will be deleted rather than marked as 'off'. We therefore require an audit table to record who/when performed data deletions/updates (this is on the requirements roadmap ).
  • Loose some flexiblity as the DB will manage the relationships (FKs) between entities rather than the application (but in turn we gain performance/simplicity/db-agnostic).
  • Changes to some PI methods (paging and carrying over v4 PKs)
  • Changes to Portal URLs and object IDs (below)

Changes

The move to Doctrine will introduce some important changes. These are outlined below.

Portal URLs

  • This affects the Web Portal GUI URLs (not the PI URLs).

In PROM, each "entity" (site, service endpoint, downtime) was assigned an ID from a global pool. There is only one object 123, the GRIDOPS-GOCDB site. The following URL is used to access an entity regardless of type:

https://goc.egi.eu/portal/index.php?Page_Type=View_Object&object_id=123&grid_id=0

In v5 using Doctrine, each entity receives an ID unique only to that entity type. There is a site with ID 123, a service endpoint with ID 123 and a downtime with ID 123. The new URL will reflect the entity type:

https://goc.egi.eu/portal/index.php?Page_Type=Site&id=123

https://goc.egi.eu/portal/index.php?Page_Type=Service_Endpoint&id=123

https://goc.egi.eu/portal/index.php?Page_Type=Downtime&id=123

(...and so on for each GOCDB entity type)

GOCDB URLs.png

PI XML Output Values

In the GOCDB v4 PI there are a number of PROM specific XML values that will change when we move to Doctrine: PRIMARY_KEY, ID and GOCDB_PORTAL_URL. These are highlighted in RED below:

Get Site Fields.png

  • ID fields will NOT be consistent between v4 and v5 (each new object will get a new Doctrine ID value).
  • The GOCDB_PORTAL_URL will therefore NOT be consistent between v4 and v5.
  • The v4 PRIMARY_KEY value will be carried over only for Sites and Downtimes, see below:



Update Aug 2013 - Preserving v4 <PRIMARY_KEY> element values

  • v4 <PRIMARY_KEY> element values are carried over for Sites and Downtimes to maintain backward compatiblity (get_site, get_site_list, get_downtime)
  • No preservation of v4 <PRIMARY_KEY> element values for:
    • get_service_endpoint
    • get_ngi
    • get_service_group
    • Please check! that this does not affect your tool.

PI get_downtime method paging

  • The ‘get_downtime’ PI method supports a new ‘&page’ parameter to limit the number of returned downtimes to a manageable limit (1000 in each page).
  • The page parameter is optional. If not specified, a query returns all matching results.
  • Paging should be used with care – since pending downtimes can be deleted, it is not guaranteed that all downtimes will be returned in a uninterrupted sequence if downtimes are actively deleted in the time it takes to issue a sequence of paged-queries.
  • If you invoke ‘get_downtime’ with no query parameters to narrow your search, the query can potentially time-out/fail because too many downtimes are requested in one hit. In this scenario, consider narrowing your query with additional query parameters before using paging.
  • The 'get_downtime' method can be re-issued by the client and each time the page parameter should be incremented by one until an empty <results/> element is returned indicating no more results, e.g.

PI get_downtime method 'all_lastmonth'

PI Scope Extensions

  • Scoping is being extended in v5 so that resources from multiple projects/infrastructures can be hosted in a single gocdb instance - see GOCDB/Release4/Development/conditionalCertificationStatusRules
  • Important: the scope extensions are backward compatible and will not break clients' existing PI queries.
  • The optional PI ‘&scope’ parameter will allow a comma-sep list of values rather than a single ‘EGI’ or ‘Local’ value.
  • An empty value can be used to ignore scope filtering: ‘&scope=’
  • If ‘&scope’ is not specified, ‘EGI’ is matched by default to preserve v4 behaviour.
  • New ‘&scope_match’ parameter for methods supporting &scope:
    • ‘&scope_match=any’ matches entities that define any of the specified scopes
    • ‘&scope_match=all’ matches entities that define all of the specified scopes

Examples:

  • Return sites that define either the EGI or PROJX scope tags: 

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&scope=EGI,PROJX&scope_match=any

  • Return sites that define both the EGI and PROJX scope tags:

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&scope=EGI,PROJX&scope_match=all

  • Return sites that define the default configured scope tag (‘EGI’) in our case (scope tag not specified):

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site

  • Return sites irrespective of their scope tags (empty ‘scope=’):

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&scope=

  • Return all DTs for Services that define both PROJX+EGI scope (scope references the Service scope):

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_downtime&scope=PROJX,EGI&scope_match=all

Request for Comments

If these changes affect a system that reads from the GOCDB-PI or links to the web portal please let us know.