From EGIWiki
Revision as of 13:50, 9 August 2016 by Kewley (talk | contribs) (v5.7)
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security

Tools menu: Main page Instructions for developers AAI Proxy Accounting Portal Accounting Repository AppDB ARGO GGUS GOCDB
Message brokers Licenses OTAGs Operations Portal Perun EGI Collaboration tools LToS EGI Workload Manager


Release date TBD

Hi everyone,

The time is nearing to announce the next GOCDB release is ready for testing (5.7). Before doing so, WP3/ops/you should be aware of the proposed updates as we may need to agree some changes or delay:

  • This release applies CURSOR-based paging to most of the API methods and replaces the current OFFSET-based paging that is available on the ‘get_downtime*’ methods. This is because of possible data inconsistency issues associated with offset paging over real-time data, see: https://www.sitepoint.com/paginating-real-time-data-cursor-based-pagination/
  • For consistency, most of the API methods will now *enforce* cursor-paging (a few ‘lightweight’ methods don’t need to support paging). If enforced paging causes too much disruption, then please inform us.
  • The XML results nest the <meta> element which contain links to the <next>, <prev> and <first> page of results.
  • When using the API, you need to move forward through the results by reading the ‘<meta> <next>’ links until the ‘<count>’ element hits zero (indicates no more results are available).
  • All results are now ordered by ascending ID (a pre-requisite of cursor-paging is to order by a natural or candidate sequence). NB: this changes the ‘get_downtime*’ methods which previously ordered downtimes by descending start date.
  • The max page size has been initially set to 400 which I think is a sensible default but this is subject to change/refinement. NB: the max page size value is provided in the <max_page_size> element.

Here are some API queries to get you started (note that the gocdb-test data is a few weeks stale and will be updated periodically):

Any issues/queries please let us know.


GOCDB team


Use of eppn in EGI

Data Privacy and Code of Conduct Policy Document Under Construction: GOCDB/data_privacy

Hi Ops/all,

Apologies for the lengthy text:

With the plans to include federated identity, services like GocDB (which falls under the R&S REFEDS SP Category) will need to use an attribute supplied by an IdP to create/identify a user account, usually ‘eduPersonPrincipleName’ (eppn) (or less likely ‘eduPersonTargetedID’, eptid, see discussion below). These attributes will be used instead of an x509 DN and will require changes to the output of the PI, changing CERTDN element to PRINCIPAL element in queries like get_user and get_site_contacts - GocDB needs to republish these attributes in its PI for use by the rest of the infrastructure (e.g. queried by Ops portal, used in accounting and so on).

Now, according to the rules of the UK Access Fed (similar rules apply in other feds), “The Service Provider must not disclose to third parties any Attributes other than to any data processor of the service provider or where the relevant end user has give its prior informed consent to such disclosure. see [note2]” . (Where [note2] states: “The basic Rule is that attributes may only be used by the service requested by the user and only for the specified purposes. Service Providers that wish to use attributes in other ways (for example to provide direct user support) can arrange this either by obtaining positive informed consent from each individual End User, or by contract with Identity Providers who are then responsible for informing their End User.”)

Therefore, to use Attributes like the eppn in EGI we need to obtain ‘positive informed consent’ from the user that we can publish and make visible their data/eppn to authenticated users and client-services of GOCDB, including those trusted by UK Access Management Federation, EGI-SSO or IGTF (I think we must explicitly state the different authentication/trust realms, e.g. any person/host with a valid IGTF cert can query the GocDB API for this info).

  • Q. Any experts out there know if the sample notification/approval-dialog shown in the screen grab below provides adequate positive consent?
  • Q. Assuming user clicks OK, do you know if there are any issues for EGI and for the hosting institution (STFC) in re-publishing the eppn in the PI? (it then becomes readable to any other service/user who has a valid IGTF cert)
    • Update 20/08/2015: Re-publishing the eppn looks like a no-go: The SP needs to first take account of each/every IdP's policy on whether the eppn can be reused. This is does not scale. However, hashing the ePPN or using the ePUID may be ok (see below).

Using ePTID without ePPN (not suitable)

GocDB could use the ‘eduPersonTargetedID’ (eptid) and re-publish that in the PI – the eptid is less of a security concern because its anonymous/opaque. However, since the eptid will be different for the same user across different SAML Service Providers, this provides little use when there is requirement to identify a particular accountID across different SPs and organisations (this is the EGI requirement). Note, if EGI had a single centralised/proxy IdP, it could be used to re-issue the same eptid consistently to all the underlying EGI services/systems. I think this would be far preferable compared to services individually registering with federations and receiving different eptids per service.

  • Update 20/08/2015, using the eptid is not suitable as it does not match the EGI requirement

Proposed Solution Requires a) and b)

a) Hashing the ePPN using a project-owned secret salt; a way to securely re-publish and correlate an ID dervied from the ePPN within a project

A group of related Service Providers, e.g. those belonging to a particular project like EGI, may concatenate a secret ‘salt’ value to the value of an attribute before hashing the result, for example; hash(‘secretSaltValue + eduPersonPrincipleName’). In doing this, the same hash value can only be reproduced if: a) an SP has access to the same secret salt value and, b) the SPs agree to use the same hashing algorithm. If the salted and hashed attribute is the ePPN, this produces an opaque and persistent identifier that can only be reproduced and correlated across the SPs within the project (i.e. those that have secure access to the salt). Provided the salt value is not disclosed to any third parties, the resulting hash string can be safely republished, for example, for use in correlating account IDs across project-specific SPs in the same project.

b) Requesting both ePPN and ePTID to determine ePPN re-allocation

The ePPN is re-assignable, and re-assignment practices depend on the federation/country; you cannot always rely on a federation re-assigning an ePPN after an agreed period of inactivity (e.g. 2yrs with the UKAMF). Therefore, as suggested by the R&S REFEDS SP Category, its good practice to also request/release both ePPN and ePTID together. This is useful because the SP can also record an ePTID change (by definition the ePTID can never be re-assigned). By doing this, the SP knows that the holder of the ePPN has changed. Note, the ePTID would only be used internally by a particular SP to check that the user has not changed; it wouldn't be used when hashing the ePPN with a secret salt for use in republishing as described above.

  • Q. I think this could be used by SPs within EGI allowing accounts to be safely correlated across SPs.

I’d be grateful for any thoughts/guidance on these matters (many thanks to Mikael Linden and Andrew Cormack for their help).




v5.5 Release

  • Test instance: https://gocdb-test.esc.rl.ac.uk/portal
  • Release date was Wed 2nd Dec.
  • The 'get_site_security_info' method will be moved from 'private,' which requires the DN of each calling client to be manually registered in an ACL (access control list), to 'protected,' which allows anyone with a valid IGTF certificate to query the method. The only info regarded as potentially sensitive is each NGI's CSIRT email and Tel. The reasons why this method was originally made private were a) because it collates all the contacts into a single query and so was largely a spam concern, b) inherited legacy - it's simply always been this way. I also understand that the original intention was that the generic CSIRT email addresses/Tels were never intended to be restricted.
  • Adding <SCOPES/> element to the XML output of the following PI queries: get_site, get_service, get_service_endpoint, get_ngi, get_service_group
  • Adding new scope tags for WLCG VOs, including ‘wlcg, atlas, cms, lhcb, alice, tier1, tier2’ for finer-grained resource filtering.
  • Large scale refactoring of the role model to cater for multi-tenant (multiple projects) and to more easily add/refine new roles and actions/permissions per-project. The main change is that roles apply to a particular project – this is shown in the output of the ‘get_user’ method that qualifies which user-roles apply to which project using the ‘<RECOGNISED_IN_PROJECTS>’ element.

    <EGEE_USER ID="1484G0" PRIMARY_KEY="1484G0">
	<TEL>+44 1925 603762</TEL>
	<CERTDN>/C=UK/O=eScience/OU=CLRC/L=DL/CN=david meredith</CERTDN>
	    <USER_ROLE>Service Group Administrator</USER_ROLE>
	    <RECOGNISED_IN_PROJECTS/>            <!-- ***new element (ServiceGroup role is Project agnostic)*** -->
	    <USER_ROLE>Chief Operations Officer</USER_ROLE>
	    <RECOGNISED_IN_PROJECTS>                    <!-- ***new element*** -->
		<PROJECT ID="1505">StandaloneProjectX</PROJECT>
	    <USER_ROLE>Site Operations Manager</USER_ROLE>
	    <RECOGNISED_IN_PROJECTS>                  <!-- ***new element*** -->