Difference between revisions of "GOCDB/Input System User Documentation"

From EGIWiki
Jump to: navigation, search
(Roles)
(Adding new services types)
 
(316 intermediate revisions by 14 users not shown)
Line 1: Line 1:
 
{{Template:Op menubar}}
 
{{Template:Op menubar}}
{{Template:Tools menubar}}
+
{{Template:GOCDB_menubar}}
<<Back to [[GOCDB/Documentation_Index]] <br/>
+
{{TOC_top}}
__TOC__
+
[[Category:GOCDB]]
 
= Introduction =
 
= Introduction =
  
 
== Scope of this documentation ==
 
== Scope of this documentation ==
This user documentation is about the GOCDB4 Input System, which is either:
+
This user documentation is about the GOCDB5 Input System, which is either:
 
* The regionally deployed instance of GOCDB, containing local information
 
* The regionally deployed instance of GOCDB, containing local information
 
* The centrally hosted instance that allows users of non regionalised NGIs to update their information
 
* The centrally hosted instance that allows users of non regionalised NGIs to update their information
 +
* For more details see: https://wiki.egi.eu/w/images/d/d3/GOCDB5_Grid_Topology_Information_System.pdf
  
 
== Other documentation ==
 
== Other documentation ==
Line 16: Line 17:
 
== Version and improvements ==
 
== Version and improvements ==
  
This documentation is meant to be useful and accurate. If you think it is not, please send us any improvement suggestions to [mailto:gocdb-admins_at_mailtalk.ac.uk gocdb-admins_at_mailtalk.ac.uk]
+
This documentation is meant to be useful and accurate. If you think it is not, please send us any improvement suggestions to [mailto:gocdb-admins@mailman.egi.eu gocdb-admins_at_mailman.egi.eu]
  
'''''GOCDB version supported in this documentation: 4.3 (April 2012)'''''
+
'''''GOCDB version supported in this documentation: 5.3+ '''''
 
<br/><br/>
 
<br/><br/>
  
 
= Quick Orientation guide =
 
= Quick Orientation guide =
  
== Accessing GOCDB4 input system ==
+
== Accessing GOCDB5 input system ==
  
 
To access the web interface, you need an '''X509 digital certificate''' installed in your browser, delivered by one of the recognised EU-Grid-PMA Certification Authorities.
 
To access the web interface, you need an '''X509 digital certificate''' installed in your browser, delivered by one of the recognised EU-Grid-PMA Certification Authorities.
  
 
* [http://www.eugridpma.org/members/worldmap/ Obtain a X509 digital certificate]
 
* [http://www.eugridpma.org/members/worldmap/ Obtain a X509 digital certificate]
* Enter GOCDB4 central web portal at https://next.gocdb.eu/portal
+
** Please note, <b>GOCDB does not support single or double quotes in the certificate DN (Distinguished Name)</b>.
 +
*** This DN is rejected by GOCDB because of the single quote: <tt>/C=UK/O=STFC/OU=SomeOrgUnit/CN=David Mc'Donald</tt>
 +
*** This is in accordance with RFC1778 which also disallows single quotes in all Relative Distinguished Name (RDN) components, and the [https://forge.ogf.org/sf/projects/caops-wg OGF Certificate Authority Working Group (CAOPS)] who strongly discourage any type of quote in a certificate DN as specified by their Grid Certificate Profile document. 
 +
* Enter GOCDB5 central web portal at https://goc.egi.eu/portal
  
 
You can access the system as soon as you have a recognised X509 certificate, however you will only be able to update information if you register and obtain a '''role'''. More information about roles and associated permission is available in the [[#Users and roles]] section.
 
You can access the system as soon as you have a recognised X509 certificate, however you will only be able to update information if you register and obtain a '''role'''. More information about roles and associated permission is available in the [[#Users and roles]] section.
Line 36: Line 40:
 
== How is the information organised? ==
 
== How is the information organised? ==
  
The following sets of information in GOCDB4 are organised in a similar way to GOCDB3:
+
GOCDB5 supports multiple projects. Each Project groups zero or more NGIs. An NGI groups zero or more Sites. A Site groups zero or more Services. ServiceGroups can also be used to group Services belonging to different Sites. Downtimes are declared over Services. Users have roles over target objects.
  
* '''Sites''' and related information
+
* '''Projects''' group child NGI's
* '''Service endpoints''' and related information
+
* '''NGIs''' group chid Sites
* '''Groups''' (NGIs, ROCs, Countries) and related information
+
* '''Sites''' group child Services
* '''Users '''and related information
+
* '''Services''' group child Downtimes
 
* '''Downtimes''' and related information
 
* '''Downtimes''' and related information
 +
* '''Service Groups''' Group Servies defined across different Sites
 +
* '''Users ''' own zero or more Roles
 +
* '''Roles''' link a User to a target entity and a defined role type.
  
 +
<br/>
 +
For more details see: https://wiki.egi.eu/w/images/d/d3/GOCDB5_Grid_Topology_Information_System.pdf
  
Main changes between GOCDB3 and GOCDB4 are related to:
+
= Users and roles =
 +
== Understanding and manipulating user accounts ==
  
* '''Nodes''' and related information: the notion of node disappears in GOCDB4 and is replaced by the notion of "service endpoints"
+
===Authentication===
* '''Groups ''' in GOCDB4 are a generic replacement of specific groups such as ROC or Country.
 
  
<br/><br/>
+
The GOCDB UI attempts to authenticate you in one of two ways (the REST style API applies x509 only):
 +
* First, by requesting an IGTF accredited user certificate from your browser. If a suitable certificate is detected, you will be asked to confirm selection of your certificate in your browser.
 +
* Second, if you do not have a user certificate or you hide your certificate from GOCDB (e.g. by starting a new/anonymous private browser session or pressing 'Cancel' when prompted for a certificate), you will be redirected to the EGI Identity Provider Service (IdP) where you can authenticate with your chosen institution (if available). If authentication is successful, you will be re-directed back to GOCDB. Please note, not all logins available in the EGI IdP provide a sufficient level of assurance (LoA) to login to GOCDB (the LoA must be 'Substantial'). 
  
= Users and roles =
+
Each GOCDB user account is linked to a single account by an ID string - this ID from comes '''either''' your Certificate DN or from the EGI IdP service. It is important to note that GOCDB does not perform account-linking - '''each ID string maps to a separate GOCDB account'''.  Existing users who have already registered an account will be logged into their account, while new users may choose to register a new account.
== Understanding and manipulating user accounts ==
 
  
 
=== Registering a new user account ===
 
=== Registering a new user account ===
  
Any new users that would like a GOCDB account have to follow this procedure. Having a grid certificate installed in your browser is enough to have read-only access to all the public features of GOCDB. If you need to edit data in GOCDB '''you will need to fill in the registration form'''.  
+
Being authenticated in one of the two ways described above is enough to have read-only access to all the public features of GOCDB. If you need to edit data in GOCDB and request roles, '''you will need to fill in the registration form'''.  
 
+
 
'''To Register:'''
 
'''To Register:'''
  
* Go to the GOCDB input system web portal  
+
* Go to the GOCDB web portal  
* In the sidebar, look out for the '''User status''' panel
+
* In the left sidebar, look out for the '''User status''' panel
* click on the "register a new account" link
+
* click on the "Register" link
 
* fill in the form and validate
 
* fill in the form and validate
  
'''Note''': If you were registered in GOCDB but are not recognised anymore (e.g. because your certificate DN changed), do not register again! Instead, follow the steps described in the [[#Changing_your_certificate_DN]] section
+
'''Note''': If you were registered in GOCDB but are not recognised anymore (e.g. because your certificate DN changed), do not register again! Instead, follow the steps described in the [[#Changing_your_accountID]] section
  
 
=== Editing your user account ===
 
=== Editing your user account ===
Line 79: Line 89:
 
There is currently no facility for listing all users in the database. List of users that have a role on a given site appears on site details pages (see section about sites). It is also possible to search for a user's account using the '''search''' feature on the sidebar.
 
There is currently no facility for listing all users in the database. List of users that have a role on a given site appears on site details pages (see section about sites). It is also possible to search for a user's account using the '''search''' feature on the sidebar.
  
=== Deactivating a user account ===
+
=== Deleting your user account ===
 
If you wish to unregister from GOCDB, follow these steps:
 
If you wish to unregister from GOCDB, follow these steps:
 
* click on the "view details" link in the "User Status" panel on the sidebar. you should get a page showing your user account information
 
* click on the "view details" link in the "User Status" panel on the sidebar. you should get a page showing your user account information
Line 85: Line 95:
 
* confirm your choice
 
* confirm your choice
  
Your account will then be deactivated and all your roles revoked.
+
Your account will then be deleted along with any roles the account has.
 +
 
 +
=== Changing your accountID ===
 +
Under the following circumstances it is possible to lose access to a GOCDB account that was originally created using a client certificate:
 +
* If you change your certificate, it is possible that the certificate's distinguished name (DN) has also changed. This is what GOCDB uses to identify your account.
 +
* If you choose to stop using your client certificate to log into GOCDB and istead access GOCDB via the EGI IdP.
 +
* If you have an account linked to your certificate but later login via the EGI-IdP route and mistakenly change your accountID from your certDN to the newly assigned ID issued by the EGI IdP.   
 +
 
 +
In these situations, it is usually possible to regain access using to your certificate based account by following one of the following procedures: 
 +
 +
====If you have a new certificate and have lost access to your account====
 +
 
 +
* First install your new certificate in your browser.
 +
* Go to GOCDB. If you are already logged in, then clear your caches and restart your browser or start a new private browser session.
 +
* When prompted, select your new certificate but <b>DON'T Register</b> a new account.
 +
* You should be able to access GOCDB, but since you are authenticated with your new certificate, it is as if you had no user account (you have not registered your new certificate with GOCDB yet).
 +
* In the "user status" panel in the sidebar, click on the '''retrieve an old account''' link.
 +
* Specify in the form the DN of your old certificate, and the e-mail address associated to your account.
 +
* Upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account. This is to avoid identity theft. The e-mail contains a validation link.
 +
* Click on the validation link or copy/paste in your browser. Once validated, changes are immediate.
 +
 
 +
====If you choose to stop using a client certificate in favour of the EGI IdP====
 +
NOTE: Following this process will mean you can *only* login to your GOCDB account via EGI Check-In going forward
 +
* Access GOCDB via the EGI IdP
 +
* In the "user status" panel in the sidebar, click on the retrieve an old account link.
 +
* Specify in the form: the DN of your old certificate; and the e-mail address associated to your account.
 +
* Upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account.
 +
* Click on the validation link or copy/paste in your browser. Once validated, changes are immediate.
 +
 
 +
====If you mistakenly changed your accountID from your certDN to the ID issued from the EGI IdP and have lost access using your certificate====
 +
 
 +
* Go to GOCDB. If you are already logged in, then clear your caches and restart your browser or start a new private browser session.
 +
* When prompted, select your certificate you want to reinstate/re-associate with your account - <b>DON'T Register</b> a new account.
 +
* You should be able to access GOCDB but since you are authenticated with the certificate that is no longer linked to your account, it is as if you had no user account. 
 +
* In the "user status" panel in the sidebar, click on the '''retrieve an old account''' link.
 +
* In the form, specify the DN of your certificate that you want to reinstate, and the e-mail address associated to your account.
 +
* Upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account. This is to avoid identity theft. The e-mail contains a validation link.
 +
* Click on the validation link or copy/paste in your browser. Once validated, changes are immediate.
 +
 
 +
 
 +
If for any reason you were unable to complete these steps (e.g. mail confirmations problems) '''please do not register a new user account''', but contact the GOCDB support helpdesk instead.
  
 
== Understanding and manipulating roles ==
 
== Understanding and manipulating roles ==
Line 100: Line 150:
 
* D' role: Users with a management role at regional level (NGI manager or deputy, security officer)
 
* D' role: Users with a management role at regional level (NGI manager or deputy, security officer)
 
* E role:  Users with a role at project level
 
* E role:  Users with a role at project level
 +
 +
The only difference between C and C' users is that:
 +
* C can NOT approve/reject role requests. 
 +
* C' can only approve/reject role requests for their SITE.
 +
The difference between D and D' users is that:
 +
* D can NOT add/delete sites to/from their NGI.
 +
* D can NOT update the certification status of member sites.
 +
* D can NOT approve or reject role requests.
  
 
==== Roles ====
 
==== Roles ====
* '''At site level'''
+
* '''At Site level'''
 
** ''Site Administrator'' - person responsible of maintaining a grid site and associated information in GOCDB (C Level)
 
** ''Site Administrator'' - person responsible of maintaining a grid site and associated information in GOCDB (C Level)
 
** ''Site Security officer'' - official security contact point at site level (C' Level)
 
** ''Site Security officer'' - official security contact point at site level (C' Level)
 
** ''Site Operations Deputy Manager'' - The deputy manager of operations at a site (C' Level)
 
** ''Site Operations Deputy Manager'' - The deputy manager of operations at a site (C' Level)
 
** ''Site Operations Manager'' - The manager of site operations (C' Level)
 
** ''Site Operations Manager'' - The manager of site operations (C' Level)
* '''At regional level'''
+
* '''At NGI/Regional level'''
** ''Regional Manager'' and ''deputy Regional manager'' - people that officially carry on NGI/Regional management
 
 
** ''Regional First Line Support'' -  Staff providing first line support for an NGI (D Level)
 
** ''Regional First Line Support'' -  Staff providing first line support for an NGI (D Level)
 
** ''Regional Staff (ROD)'' - staff involved in [[Glossary#Operations_Centre|Operations Centre]] activities such as user/operations support (D Level)
 
** ''Regional Staff (ROD)'' - staff involved in [[Glossary#Operations_Centre|Operations Centre]] activities such as user/operations support (D Level)
Line 114: Line 171:
 
** ''NGI Operations Deputy Manager'' - Deputy manager of NGI operations (D' Level)
 
** ''NGI Operations Deputy Manager'' - Deputy manager of NGI operations (D' Level)
 
** ''NGI Operations Manager'' - Manager of NGI operations (D' Level)
 
** ''NGI Operations Manager'' - Manager of NGI operations (D' Level)
* '''At project level (E Users)'''
+
* '''At Project level'''
** ''Chief Operations Officer (COO)'' EGI
+
** ''COD staff'' - COD staff (E Level)
** ''COD staff'' - COD staff (as previously defined in EGEE-III operational model - need EGI equivalent)
+
** ''COD administrator'' - People administrating Central COD roles (E Level)
** ''C-COD administrator'' - People administrating Central COD roles
+
** ''EGI CSIRT Officer'' - official security contact point at project level (E Level)
** ''EGI CSIRT Officer'' - official security contact point at project level
+
** ''Chief Operations Officer (COO)'' - The EGI Chief Operations Officer (E Level)
  
=== Permissions associated to roles ===
+
=== Permissions associated to roles ===
  
GOCDB roles and permissions are based on whether the considered object is owned or not. In the table below the following definitions apply:
+
GOCDB roles and permissions are based on whether the considered object is owned or not. In the table below the following definitions apply:  
* '''Owned group''': a group on which the role applies (ROC, NGI, project)
 
* '''Owned site''': a site on which the role applies, or belonging to an owned group
 
* '''Owned service endpoint''': a service endpoint belonging to an owned site
 
  
Each role has a set of associated permissions which apply on the role's scope (site, region or project). Main permissions are summarised in the table below
+
*'''Owned group''': a group on which the role applies (ROC, NGI, project)
<!--
+
*'''Owned site''': a site on which the role applies, or belonging to an owned group
 +
*'''Owned service endpoint''': a service endpoint belonging to an owned site
 +
 
 +
Each role has a set of associated permissions which apply on the role's scope (site, region or project). Main permissions are summarised in the table below <!--
 
, and grouped by user categories as follows:
 
, and grouped by user categories as follows:
 
* '''A users''': Unregistered users
 
* '''A users''': Unregistered users
 
* '''B users''': Registered users with no role
 
* '''B users''': Registered users with no role
 
* '''C users''': Users with a role at site level (site admin, site security officer...)
 
* '''C users''': Users with a role at site level (site admin, site security officer...)
* '''D users''': Users with a role at regional level (NGI manager or deputy, regional operations staff...)
+
* '''D users''': Users with a role at NGI/regional level (NGI manager or deputy, regional operations staff...)
 
* '''E users''': Users with a role at project level
 
* '''E users''': Users with a role at project level
-->
+
--> <br>  
<br clear="right" />
+
 
{| class="wikitable" border="1" cellspacing="0" cellpadding="3"
+
{| cellspacing="0" cellpadding="3" border="1" class="wikitable"
 
|-
 
|-
 
! Action  
 
! Action  
! A) Unregistered users
+
! A) Unregistered users  
! B) Registered users with no role
+
! B) Registered users with no role  
! C) Site level users
+
! C) Site level users  
! D) Regional level users
+
! C' ) Site Management Level Users<br>
 +
! D) NGI level users  
 +
! D' ) NGI Management Level Users
 
! E) Project level users
 
! E) Project level users
 
|-
 
|-
|Add a site to an owned group|| ''irr.'' || ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || ''irr.''
+
| Add a site to an owned group  
|-
+
| ''irr.''  
|Add a site to a non owned group|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| ''irr.''  
|-
+
| ''irr.''  
|Add a service endpoint to an owned site|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''  
+
| ''irr.''  
|-  
+
| style="color:red" | no  
|Add a service endpoint to a non owned site|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color: green" | '''yes'''  
|-
+
| ''irr.''
|Add a downtime to an owned service endpoint|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''
+
|-
|-  
+
| Add a site to a non owned group
|Add downtime to a non owned service endpoint|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no  
|-
+
| style="color:red" | no  
|Update information of an owned site|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''
+
| style="color:red" | no  
|-
+
| style="color:red" | no  
|Update information of a non owned site|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no  
|-
+
| style="color:red" | no
|Update certification status of an owned site|| ''irr.'' || ''irr.'' || style="color:red" | no|| style="color:green" | '''yes''' || ''irr.''
+
| style="color:red" | no
|-
+
|-
|Update certification status of a non owned site|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:green" | '''yes'''
+
| Add a service endpoint to an owned site  
|-
+
| ''irr.''  
|Update information of a owned service endpoint|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''  
+
| ''irr.''  
|-  
+
| style="color:green" | '''yes'''  
|Update information of a non owned service endpoint|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:green" | '''yes'''  
|-
+
| style="color:green" | '''yes'''  
|Update information of an owned group|| ''irr.'' || ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || ''irr.''
+
| style="color:green" | '''yes'''  
|-
+
| ''irr.''
|Update information of a non owned group|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
|-
|-
+
| Add a service endpoint to a non owned site
|Update own user account details|| ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes'''
+
| style="color:red" | no  
|-
+
| style="color:red" | no  
|Update other user's account|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no  
|-  
+
| style="color:red" | no  
|Update a downtime on an owned service endpoint|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''
+
| style="color:red" | no  
|-
+
| style="color:red" | no  
|Update a downtime on a non owned service endpoint|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no
|-
+
|-
|Delete an owned site|| ''irr.'' || ''irr.'' ||  no || style="color:red" | no|| style="color:red" | no
+
| Add a downtime to an owned service endpoint  
|-
+
| ''irr.''  
|Delete a non owned site|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| ''irr.''  
|-
+
| style="color:green" | '''yes'''  
|Delete an owned service endpoint|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''
+
| style="color:green" | '''yes'''  
|-
+
| style="color:green" | '''yes'''  
|Delete a non owned service endpoint|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:green" | '''yes'''  
|-
+
| ''irr.''
|Delete an owned group|| ''irr.'' || ''irr.'' || ''irr.'' || style="color:red" | no|| ''irr.''  
 
 
|-
 
|-
|Delete a non owned group|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| Add downtime to a non owned service endpoint
|-
+
| style="color:red" | no  
|Delete a downtime on an owned service endpoint|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''  
+
| style="color:red" | no  
|-  
+
| style="color:red" | no  
|Delete a downtime on a non owned service endpoint|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no  
|-  
+
| style="color:red" | no  
|Delete your own user account|| ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes'''  
+
| style="color:red" | no
|-  
+
| style="color:red" | no
|Delete other user's account|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
|-
|-
+
| Update information of an owned site
|Register a new user account|| style="color:green" | '''yes''' || ''irr.'' || ''irr.'' || ''irr.'' || ''irr.''  
+
| ''irr.''  
|-
+
| ''irr.''  
|Request a new role|| style="color:red" | no|| style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes'''
+
| style="color:green" | '''yes'''  
|-
+
| style="color:green" | '''yes'''  
|Approve a role request on an owned site or group|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''  
+
| style="color:green" | '''yes'''
|-
+
| style="color:green" | '''yes'''
|Approve a role request on a non owned site or group|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| ''irr.''
|-
+
|-
|Reject a role request on an owned site or group|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''  
+
| Update information of a non owned site
|-
+
| style="color:red" | no
|Reject a role request on a non owned site or group|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no  
|-
+
| style="color:red" | no
|Revoke an existing role on an owned object|| ''irr.'' || ''irr.'' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || ''irr.''
+
| style="color:red" | no  
|-
+
| style="color:red" | no  
|Revoke an existing role on a non owned object|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no|| style="color:red" | no
+
| style="color:red" | no  
|-
+
| style="color:red" | no
|Retrieve an existing account/ change certificate DN|| style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes''' || style="color:green" | '''yes'''  
+
|-
 +
| Update certification status of an owned site
 +
| ''irr.''
 +
| ''irr.''  
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:green;" | '''yes'''  
 +
| style="color:green;" | '''yes'''  
 +
|-
 +
| Update certification status of a non owned site
 +
| style="color:red" | no  
 +
| style="color:red" | no  
 +
| style="color:red" | no  
 +
| style="color:red" | no  
 +
| style="color:red" | no  
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
|-
 +
| Update information of a owned service endpoint
 +
| ''irr.''  
 +
| ''irr.''  
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''  
 +
| ''irr.''
 +
|-
 +
| Update information of a non owned service endpoint
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Update information of an owned group
 +
| ''irr.''  
 +
| ''irr.''  
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| ''irr.''
 +
|-
 +
| Update information of a non owned group
 +
| style="color:red" | no  
 +
| style="color:red" | no
 +
| style="color:red" | no  
 +
| style="color:red" | no  
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Update own user account details
 +
| ''irr.''  
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
|-
 +
| Update other user's account
 +
| style="color:red" | no  
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no  
 +
| style="color:red" | no
 +
|-
 +
| Update a downtime on an owned service endpoint
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''  
 +
| style="color:green" | '''yes'''  
 +
| ''irr.''
 +
|-
 +
| Update a downtime on a non owned service endpoint
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Delete an owned site
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Delete a non owned site
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Delete an owned service endpoint
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| ''irr.''
 +
|-
 +
| Delete a non owned service endpoint
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Delete an owned group
 +
| ''irr.''
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| ''irr.''
 +
|-
 +
| Delete a non owned group
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Delete a downtime on an owned service endpoint
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| ''irr.''
 +
|-
 +
| Delete a downtime on a non owned service endpoint
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Delete your own user account
 +
| ''irr.''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
|-
 +
| Delete other user's account
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Register a new user account
 +
| style="color:green" | '''yes'''
 +
| ''irr.''
 +
| ''irr.''
 +
| ''irr.''
 +
| ''irr.''
 +
| ''irr.''
 +
| ''irr.''
 +
|-
 +
| Request a new role
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
|-
 +
| Approve a role request on an owned group
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color: green" | '''yes'''
 +
| style="color: green" | '''yes'''
 +
|-
 +
| Approve a role request on an owned site
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color: green" | '''yes'''
 +
| style="color:red" | no
 +
| style="color: green" | '''yes'''
 +
| ''irr''
 +
|-
 +
| Approve a role request on a non owned site or group
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Reject a role request on an owned group
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
| ''irr.''
 +
|-
 +
| Reject a role request on an owned site
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
| ''irr''
 +
|-
 +
| Reject a role request on a non owned site or group
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Revoke an existing role on an owned object
 +
| ''irr.''
 +
| ''irr.''
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
| style="color:red" | no
 +
| style="color:green" | '''yes'''
 +
| ''irr.''
 +
|-
 +
| Revoke an existing role on a non owned object
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
| style="color:red" | no
 +
|-
 +
| Retrieve an existing account/ change certificate DN
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 +
| style="color:green" | '''yes'''
 
|}
 
|}
  
<br/>
+
<br>
 +
 
 +
=== Requesting roles for your account  ===
 +
 
 +
There are 2 ways to request new roles.
  
=== Requesting roles for your account ===
+
*By clicking on the '''manage role''' link (sidebar, user status panel)
 +
**the first form allows you to choose the entity (site or group) on which you want to request a role
 +
**the second form lets you choose the role you want to apply for
  
There are 2 ways to request new roles.
+
*By clicking on the '''request role''' link from site detail pages or group detail pages.  
 +
**displayed form lets you choose the role you want to apply for
  
* By clicking on the '''manage role''' link (sidebar, user status panel)
 
** the first form allows you to choose the entity (site or group) on which you want to request a role
 
** the second form lets you choose the role you want to apply for
 
<br/>
 
* By clicking on the '''request role''' link from site detail pages or group detail pages.
 
** displayed form lets you choose the role you want to apply for
 
<br/>
 
 
Once made, role requests have to be validated before the role is granted to you. This part of the process is described in the next section.
 
Once made, role requests have to be validated before the role is granted to you. This part of the process is described in the next section.
  
Line 240: Line 576:
 
=== Changing your certificate DN ===
 
=== Changing your certificate DN ===
  
If you change your certificate, it is possible that the certificate's distinguished name (DN) has also changed. This is what GOCDB uses to identify your user account.
+
Moved to: [[#Changing_your_accountID]]
 
 
* '''after having installed your new certificate'''
 
** If you enter GOCDB with your new certificate it will be as if you had no user account (as GOCDB will not know your new certificate).
 
** in the "user status" panel in the sidebar, click on the '''retrieve an old account''' link
 
** specify in the form the DN of your old certificate, and the e-mail address associated to your account
 
** upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account. This is to avoid identity theft. The e-mail contains a validation link
 
** click on the validation link or copy/paste in your browser. Once validated, changes are immediate.
 
 
 
If for any reason you were unable to complete these steps (e.g. mail confirmations problems) '''please do not register a new user account''', but contact the GOCDB support helpdesk instead with your old and new certificate DNs.
 
  
 
=== Approving role  and change requests ===
 
=== Approving role  and change requests ===
Line 265: Line 592:
 
In order to approve or decline role requests, simply click on the '''accept''' or '''deny''' links in front of each role request.
 
In order to approve or decline role requests, simply click on the '''accept''' or '''deny''' links in front of each role request.
  
=== Revoking roles ===
+
=== Revoking roles ===
  
If a user within your scope has a role that needs to be revoked, you can do this from the user's page, where user's details are listed along with his/her current roles. To revoke a role, simply click on the '''view''' link in front of the concerned role, then on the '''revoke''' link at the top of the role's details page.
+
If a user within your scope has a role that needs to be revoked, you can do this from the user's page, where user's details are listed along with his/her current roles. To revoke a role, simply click on the role name then on the '''revoke''' link at the top right of the role's details page.  
  
 
'''Note''': This works for other users within your scope but also for yourself. However just note that if you revoke your own roles you may not have proper permissions to recover them afterwards.
 
'''Note''': This works for other users within your scope but also for yourself. However just note that if you revoke your own roles you may not have proper permissions to recover them afterwards.
  
<br/><br/>
+
= NGIs (Site Group)=
 +
An NGI forms a grouping of Sites in GOCDB. GOCDB stores the following information about these groups. 
 +
The main page listing groups actually shows NGIs/ROCs, and is available from
 +
* '''List of NGIs/ROCs and associated contacts''', linked from the main menu
 +
 
 +
 
 +
Each NGI has its own listing page, accessible by clicking on the "view" link in group listing pages. A group details page shows users with a role on that group, as well as member sites and associated contacts and roles.
 +
 
 +
== Adding NGIs ==
 +
 
 +
Adding groups is not possible through the Input System web interface.
 +
If you want to start the registration process of a new NGI, please follow the procedure described on:
 +
* [[PROC02|Operations Centre creation procedure]]
 +
 
 +
Integration of the new group in GOCDB is part of the procedure but has to be done by GOCDB admins.
 +
 
 +
== Editing Groups ==
 +
To edit a group, simply click on the "edit" link at the top of the group's details page.
 +
 
 +
 
 +
== Deleting Groups ==
 +
This operation is not allowed.
 +
 
  
 
= Sites =
 
= Sites =
 
== Definition ==
 
== Definition ==
A site is a grouping of grid resources collating multiple Service Endpoints (SEs). Down times are recorded on selected SEs of a site. GOCDB stores the following information about sites (non exhaustive list):
+
A site (also known as a Resource Centre) is a grouping of grid resources collating multiple Service Endpoints (SEs). Down times are recorded on selected SEs of a site. GOCDB stores the following information about sites (non exhaustive list). Note, when editing values in the portal, mandatory fields are marked with '*':
  
* A unique (short) site name
+
* A unique (short) name - case sensitive (GOCDB and GoCDB are considered different)
* An official (long) site name
+
* An official (long) name
* A domain name for the site
+
* A domain name for the Site/Resource Centre
* The home web URL of the site
+
* The home web URL of the Site/Resource Centre
 
* A contact email address and telephone number
 
* A contact email address and telephone number
 
** Emergency e-mail for a fast response time in case of urgent problem
 
** Emergency e-mail for a fast response time in case of urgent problem
Line 286: Line 635:
 
* A security contact email address and telephone number
 
* A security contact email address and telephone number
 
* The site timezone
 
* The site timezone
* The site's GIIS URL
+
* The site's GIIS URL (Case Sensitive - Please ensure you enter your Site name which is usually encoded in the URL in the correct case!).
* A description of the site
+
** e.g. ldap://bdii-rc.some-site.uk:2170/mds-vo-name=SITE-NAME,o=grid  (if your GOCDB site name site name is upper case)
 +
* A mandatory human readable description of the site
 
* The site's latitude, longitude and location
 
* The site's latitude, longitude and location
* PRODUCTION_STATUS [GROUP] The site's intended production infrastructure, has one of the following values:  
+
* Production Infrastructure: The site's intended target infrastructure. This specifies the infrastructure that the site's services deliver to. This has one of the following values:  
** Production  
+
** Production (with this target infrastructure, the EGI site certification transition rules apply)
** Pre-production (PPS)  
+
** Test (in future, if the site delivers to this infrastructure, then its Certification status will be fixed to 'Candidate').
** Test  
 
** <strike>SC</strike> Obsolete
 
 
* ROC [GROUP] - The NGI or Region of the site  
 
* ROC [GROUP] - The NGI or Region of the site  
 
* Country
 
* Country
 +
* IP address range within which the Site/Resource Centre's services run
 +
** IP/netmask (x.x.x.x/x.x.x.x). To specify multiple IP/netmask values, use a comma or semi-colon separated list with no spaces, e.g. 1.2.3.4/255.255.255.0,1.2.3.5/255.255.255.0
  
 
== Manipulating sites ==
 
== Manipulating sites ==
Line 315: Line 665:
 
=== Editing site information ===
 
=== Editing site information ===
 
The editing process will show you the same form as the adding process. To edit a site, simply click the "'''edit'''" link on top of the site's details page.
 
The editing process will show you the same form as the adding process. To edit a site, simply click the "'''edit'''" link on top of the site's details page.
 +
 +
===Renaming a site===
 +
Provided you have permissios, you can change the Short Name, Official Name and GIIS URL to the new Resource Center details.
 +
For more information regarding the site renaming procedure please see: [[PROC15 ]]
  
 
=== Removing a site ===
 
=== Removing a site ===
Line 323: Line 677:
  
  
For each site, GOCDB stores and shows information about its certification status. This reflects the different steps of the official SA1 site certification procedure which typically follows:  
+
For each site that delivers to the 'Production' Target Infrastructure, GOCDB stores and shows information about its certification status. This reflects the different steps of the official SA1 site certification procedure which typically follows:  
 
* Candidate -> Uncertified -> Certified.  
 
* Candidate -> Uncertified -> Certified.  
 
The different possible certification statuses are:
 
The different possible certification statuses are:
Line 358: Line 712:
 
More information about site certification statuses can be found in SA1 certification and operation procedures documents:
 
More information about site certification statuses can be found in SA1 certification and operation procedures documents:
  
* [https://wiki.egi.eu/wiki/Operations_Manuals#Operational_Procedures EGI Operational Procedures Manual]
 
 
* [[PROC09| Resource Centre Registration and Certification Procedure]]
 
* [[PROC09| Resource Centre Registration and Certification Procedure]]
  
 
<!--* [https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEROperationalProcedures View EGEE SA1 operational procedures manual (need EGI equivalent)]-->
 
<!--* [https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEROperationalProcedures View EGEE SA1 operational procedures manual (need EGI equivalent)]-->
* [[Operational_Procedures|EGI Operations Procedures]]
+
* [[Operations_Procedures|EGI Operations Procedures]]
<!--* [https://wiki.egi.eu/wiki/PROC11draft Decommissioning good practices between Site and User].  -->
 
 
*[[PROC11|Resource Centre Decommissioning Procedure]]
 
*[[PROC11|Resource Centre Decommissioning Procedure]]
 
*[[PROC12|Production Service Decommissioning Procedure]]
 
*[[PROC12|Production Service Decommissioning Procedure]]
Line 370: Line 722:
  
 
<br/><br/>
 
<br/><br/>
 +
 +
=== Defining Pay4Use Properties ===
 +
* GOCDB is used to define properties for the EGI Pay for Use pilot project. Please see: 
 +
* [https://wiki.egi.eu/wiki/EGI_Pay-for-Use_PoC:Processes/Instructions P4U instructions]
 +
* [https://wiki.egi.eu/wiki/EGI_Pay-for-Use_PoC:Service/Price_Overview price overview]
  
 
= Service Endpoints =
 
= Service Endpoints =
Line 391: Line 748:
 
* ''myhost.domain.orgUI URL: http://myhost.domain.org/UI''
 
* ''myhost.domain.orgUI URL: http://myhost.domain.org/UI''
 
* ''myhost.domain.orgunicore6.UNICOREX URL: http://myhost.domain.org/UnicoreX''
 
* ''myhost.domain.orgunicore6.UNICOREX URL: http://myhost.domain.org/UnicoreX''
 +
 +
Note that a single host can also specify multiple services of the same service type.
  
 
== Manipulating service endpoints ==
 
== Manipulating service endpoints ==
Line 425: Line 784:
  
 
This indicates whether the service is a beta service or not (part of the staged rollout process). Beta is the equivalent at service level of the former EGEE Pre-Production Service (PPS)
 
This indicates whether the service is a beta service or not (part of the staged rollout process). Beta is the equivalent at service level of the former EGEE Pre-Production Service (PPS)
 +
 +
=== Host DN ===
 +
This is the DN of the host certificate for the service. The format of the DN follows that defined by the [[https://www.ogf.org/documents/GFD.225.pdf OGF Interoperable Certificate Profile]] which restricts allowed chars to a PrintableString that does NOT contain characters that cannot  be  expressed  in  printable  7-bit  ASCII. For a list of allowed chars, see GFD.225.
 +
 +
To supply multiple or alternate DN(s) for a service, for example of the multiple hosts supporting a single service entry, see https://wiki.egi.eu/wiki/GOCDB/Input_System_User_Documentation#HostDN
  
 
=== "production" flag (t/f)===
 
=== "production" flag (t/f)===
The SEs Production flag indicates if this service delivers a production quality service to the infrastructure it belongs to (EGI). Even if this flag is false, the service is still considered part of the EGI and so shows up in the dashboard. This is not to be confused with '''PRODUCTION_STATUS''', which is a Site level flag that shows if the site delivers to the production, pre-production (PPS) or test infrastructure.
+
The services Production flag indicates if this service delivers a production quality service to the infrastructure it belongs to (EGI).  
 +
* Non-production services can be either Monitored or Not Monitored, depending on the Administrator's choice.
 +
* Even if this flag is false, the service is still considered part of the EGI and so shows up in the dashboard.  
 +
* If true, then the Monitored flag must also be true: <b>All production resources MUST be monitored</b> (except if the service type is a VOMS or emi.ARGUS)
 +
* This flag is not to be confused with '''PRODUCTION_STATUS''', which is a Site level flag that shows if the site delivers to the production or Test infrastructure.
  
 
=== "monitoring" flag (t/f)===
 
=== "monitoring" flag (t/f)===
This flag is taken into account by monitoring tools. if it is set to "N" the endpoint won't be tested.
+
This flag is taken into account by monitoring tools.  
 +
* Can only be set to "N" (false) if Production flag is also false.
 +
* If set to "N" the endpoint won't be tested.
  
=== how "production" and "monitoring" are used ===
+
=== Usage of PRODUCTION and MONITORED flags for EGI Service Endpoints ===
 +
From 02/12/2014 <b>all production services MUST be monitored</b> (except for emi.ARGUS and VOMS service types).
 +
<!--
 +
* production
 +
Monitored: YES
  
Production and monitoring are combined in the following way:
+
Comment: <b>All production resources MUST be monitored</b><br/>
* All production resources have to be monitored.
+
* <b>DM 18/09/2014:</b> <u>currently this is not enforced by a logic rule</u>, this can be easily added if required.
* All other resources can be monitored or not following site administrators' choice.
+
** Note, Production=True and Mon=False has existed since GOCDB4 as shown in the GOCDB4 archive: https://gocdb4.esc.rl.ac.uk/portal/index.php?Page_Type=Service_Endpoints&serviceType=&searchTerm=&production=Y&monitored=N&egiVisible=
 +
** So, are there reasons why production services should not _always_ be monitored? Or, is this simply an inherited legacy issue from gocdb4?
 +
*** Note, there may have once been a requirement to set Prod=T and Mon=F to prevent flooding the dashboard with alerts for a particular SAM monitoring probe that once-upon-a-time did not work with ARC and led to alerts in the monitoring system - was this the reason?
 +
** Related discussion: https://ggus.eu/index.php?mode=ticket_info&ticket_id=107363
 +
** Need to resolve this - there are quite a few current services that have Prod=True and Mon=False: https://goc.egi.eu/portal/index.php?Page_Type=Services&serviceType=&serviceTypeSearch=&ngi=&searchTerm=&production=TRUE&monitored=FALSE&scope=EGI&certificationStatus=Certified&servKeyNames=&servKeyNamesSearch=
 +
-->
  
Only test results for production+monitored services will be used for availability and reliability calculation.
+
====Production and Monitored====
 +
* Operations Dashboard: A failing test of production service endpoints generates an alarm in the ROD Operations Dashboard.
 +
* Availability calculation: The service endpoint test results are considered for Availability computation (if and only if the service type associated to the endpoint is one of those included in Availability computation)
 +
 
 +
====Non-Production and Monitored: YES/NO====
 +
* Availability calculation: If Monitored is set to YES, Service Availability Monitoring (SAM) will test the service, but SAM test results are ignored by the Availability Computation Engine (ACE).
 +
* Availability calculation: Non-production service endpoints are <u>not</u> considered for site availability calculations.
 +
* Operations Dashboard: If Monitored is set NO, the service endpoint is ignored by SAM and no alarms are raised in the Operations Dashboard in case of CRITICAL failure.
 +
* SAM tests for non-production services generate alarms into the ROD Operations Dashboard in case of CRITICAL failure of the test. These alarms are visible in the Operations Dashboard and are tagged as "non production".
  
 
= Service Groups =
 
= Service Groups =
A service group is an arbitrary grouping of existing service endpoints that can be distributed across different physical sites (also known as 'Virtual Sites'):  
+
A service group is an arbitrary grouping of existing service endpoints that can be distributed across different physical sites and users that belong to the SG (SGs were previously known as 'Virtual Sites'):  
* Each service endpoint that appears in a group <u>must already be a member of one hosting physical site</u> (a service group cannot own its own services).  
+
* Each service that appears in a group <u>must already exist and be hosted by a physical site</u>.  
* A service group role does <u>not extend any permissions</u> over its child services.  
+
* A service group role does <u>not extend any permissions</u> over its child services. This means that you cannot declare a downtime on the services that you group together or modify the service attributes. 
* Currently, any GOCDB user can create their own service groups (everything is logged, including who created the service group).  
+
* Any GOCDB user can create their own service group and as the 'Service Group Administator' you can control subsequent user membership requests to the SG (everything is logged, including who created the service group).
* Service groups are typically <u>used for monitoring a particular collection of services</u> using the GOCDB get_service_group PI method.  
+
* GOCDB users can request to join an existing service group by finding the target SG and requesting a role on that SG.  
* Service groups are a new feature. If you have any further use-cases or suggestions, please submit a ticket to RT.
+
* Service groups are typically <u>used for monitoring a particular collection of services and/or users</u> using the GOCDB 'get_service_group' and 'get_service_group_role' PI methods.
 +
* SG memebers can be listed using the get_service_group_role PI method.  
 +
* PI doc:
 +
** https://wiki.egi.eu/wiki/GOCDB/PI/get_service_group
 +
** https://wiki.egi.eu/wiki/GOCDB/PI/get_service_group_role
 +
* If you have any further use-cases or suggestions, please submit a ticket to RT.
 +
 
 +
 
 +
= NGI Core Services =
 +
NGIs can register a number of ‘NGI-Core’ services in GOCDB.  A core NGI service is one that is used to calculate the availability and reliability of the NGI. These services fall under the responsibility of the NGI and provide production quality (no testing instances).  NGIs can distinguish/flag their core services from their other (non-core) services using one of two ways (see A and B below).
 +
 
 +
=== Core Service Requirements  ===
 +
 
 +
The service instance MUST:
 +
 
 +
*Define the ‘NGI’ scope (see [[#Data_Visibility_.2F_Scopes|Data Visibilty Scopes]])
 +
*Be flagged as ‘Production’ (see [[#.22production.22_flag_.28t.2Ff.29|Production Flag]])
 +
*Not be flagged as ‘Beta’ (see [[#.22beta.22_flag_.28t.2Ff.29|Beta Flag]])
 +
*Monitored flag set to true (see [[#.22monitoring.22_flag_.28t.2Ff.29 | Monitored Flag]])
 +
*Be hosted under a ‘NGI’ scoped Site that has a certification status of ‘Certified’
 +
 
 +
=== Required Service Types  ===
 +
 
 +
The following service types are mandatory and all NGIs in the EGI scope should define instances of these services:
 +
 
 +
*'''ngi.SAM''' (Mandatory)
 +
*'''emi.ARGUS''' (Mandatory) (NGI&nbsp;ARGUS)
 +
*'''Top-BDII''' (Mandatory)
 +
 
 +
Other '''Mandatory''' services, depending on middleware deployed by sites under NGI responsibility, are listed [[NGI services in GOCDB#Services|here]]
 +
 
 +
NGIs should also register their custom core services like accounting, helpdesk if they are registered in GOCDB (for a list of other common core service types see: https://wiki.egi.eu/wiki/NGI_services_in_GOCDB)
 +
 
 +
=== Registering NGI Core Services  ===
 +
 
 +
NGI core services can be grouped/flagged in one of two ways:
 +
 
 +
*A) By creating a '''‘NGI_XX_SERVICES’ Site''' and adding their core services under this site. This site must be scoped as ‘NGI’ and define a certification status of ‘Certified’.
 +
*B) By creating a '''‘NGI_XX_SERVICES’ ServiceGroup''' and adding their core services to this [[#Service_Groups|ServiceGroup]].
 +
 
 +
<u>It is important that these core service Sites/ServiceGroups adhere to the ‘NGI_XX_SERVICES’ naming scheme</u>. For further details, including a list of existing ‘NGI_XX_SERVICES’ please see: https://wiki.egi.eu/wiki/NGI_services_in_GOCDB
  
 
= Downtimes =
 
= Downtimes =
 
== Definition ==
 
== Definition ==
A downtime is a period of time for which a grid resource is declared to be  inoperable. Downtimes may be scheduled (e.g. for software/hardware upgrades), or unscheduled (e.g. power outages). GOCDB stores the following information about downtimes (non exhaustive list):
+
A downtime is a period of time for which a service is declared to be  inoperable. Downtimes may be scheduled (e.g. for software/hardware upgrades), or unscheduled (e.g. power outages). GOCDB stores the following information about downtimes (non exhaustive list):
  
 
* The downtime classification (Scheduled or unscheduled)
 
* The downtime classification (Scheduled or unscheduled)
Line 458: Line 885:
 
* A description of the downtime
 
* A description of the downtime
 
* The entities affected by the downtime
 
* The entities affected by the downtime
 
  
 
== Manipulating downtimes ==
 
== Manipulating downtimes ==
Line 466: Line 892:
 
There are different pages on which downtimes are listed:
 
There are different pages on which downtimes are listed:
  
* '''An "archives" page''', linked from the main menu, that allows to search through all downtimes
+
* '''A "Recent and Planned" page ''', linked from the main menu, this provides a window to the EGI hosted information about downtimes. (Nb: This service requires users to add an exception to allow this window to render correctly)
* '''A "recent and planned" page ''', linked from the main menu, that presents a search tool implemented by the EGI Operations Portal
+
* '''A "Active & Imminent" page''', linked from the main menu, that allows users to see currently active downtimes and downtimes planned in the coming weeks.
 
* '''Site details page''', where all the downtimes associated to the site are listed
 
* '''Site details page''', where all the downtimes associated to the site are listed
 
* '''Service endpoint details page''', where all the downtimes associated to the service endpoint are listed
 
* '''Service endpoint details page''', where all the downtimes associated to the service endpoint are listed
 +
* '''Service group details page''', where all the downtimes associated to the service group are listed
  
 
Each downtime has its own listing page, accessible by clicking on the "view" link in downtime listing pages.
 
Each downtime has its own listing page, accessible by clicking on the "view" link in downtime listing pages.
Line 479: Line 906:
 
This is done in 2 steps:  
 
This is done in 2 steps:  
 
* enter downtime information  
 
* enter downtime information  
* specify the full list of impacted services in case there is more than one
+
* specify the full list of impacted services in case there is more than one or select an site to select all the sites associated services.
 
<br/>
 
<br/>
 
'''Please note:'''
 
'''Please note:'''
 
* All dates have to be entered in UTC.
 
* All dates have to be entered in UTC.
 +
* A downtime can be retrospectively added if its start-date is less than 48h in the past (giving a 2 day window to add).
 
* downtime classification (scheduled/unscheduled) is determined automatically (see [[#Scheduled or unscheduled?]] section)
 
* downtime classification (scheduled/unscheduled) is determined automatically (see [[#Scheduled or unscheduled?]] section)
  
 
=== Editing downtime information ===
 
=== Editing downtime information ===
  
To edit a downtime, simply click the "edit" link on top of the downtime's details page.
+
* To edit a downtime, simply click the "edit" link on top of the downtime's details page.
 
+
* A downtime can be retrospectively updated if its start-date is less than 48h in the past (giving a 2 day window to modify).
Note there are some limitations to downtime edition, especially if it has already started or is completely finished. See  [[#Downtime shortening and extension]] section for more details.
+
* Note there are limitations to downtime editing, especially if it has already started, or is due to start in the next 24hrs or is finished. See  [[#Downtime shortening and extension]] section for more details.
  
 
=== Removing downtimes ===
 
=== Removing downtimes ===
Line 498: Line 926:
  
 
=== Scheduled or unscheduled? ===
 
=== Scheduled or unscheduled? ===
depending on the planning of the intervention, downtimes can be:
+
Depending on the planning of the intervention, downtimes can be:
  
 
* '''Scheduled''': planned and agreed in advance
 
* '''Scheduled''': planned and agreed in advance
 
* '''Unscheduled''': planned or unplanned, usually triggered by an unexpected failure or at a short term notice
 
* '''Unscheduled''': planned or unplanned, usually triggered by an unexpected failure or at a short term notice
  
EGI defines precise rules about what should be declared as scheduled or unscheduled, based on '''how long in advance''' the downtime is declared. These rules are described in https://wiki.egi.eu/wiki/MAN02#How_to_manage_an_intervention
+
EGI defines precise rules about what should be declared as scheduled or unscheduled, based on '''how long in advance''' the downtime is declared. These rules are described in [[MAN02#How_to_manage_an_intervention]]
 
and are enforced as follows:
 
and are enforced as follows:
  
Line 510: Line 938:
  
 
'''Notes''':  
 
'''Notes''':  
* Unscheduled downtimes can be retroactively declared up to 48h in the past.  
+
* A downtime can be retrospectively declared and/or updated if its start-date is less than 48h in the past (giving a 2 day window to add/modify).  
 
* Although 24h in advance is enough for the downtime to be classified as "scheduled", it is good practice to declare it at least 5 working days before it starts.
 
* Although 24h in advance is enough for the downtime to be classified as "scheduled", it is good practice to declare it at least 5 working days before it starts.
 
<br/>
 
<br/>
Line 522: Line 950:
 
* '''OUTAGE''' means the resource is considered as unavailable. Such downtimes will be considered as "IN MAINTENANCE" by monitoring and availability calculation tools.
 
* '''OUTAGE''' means the resource is considered as unavailable. Such downtimes will be considered as "IN MAINTENANCE" by monitoring and availability calculation tools.
  
=== Downtime notifications ===
+
=== Downtime shortening and extension ===
  
The whole downtime notification process is described on a document available in CERN EDMS:
+
Limitation rules to downtime extensions are enforced in GOCDB as follows:
 +
* Scheduled downtimes due to start in 24 hours cannot be edited in any way, but can be deleted.
 +
* Other downtimes that have not yet started can be edit and deleted.
 +
** They can be shortened or moved, i.e. They can be edited such that:
 +
*** Both start and end time are still in the future
 +
*** The duration remains the same or is decreased
 +
* Ongoing downtimes can not be deleted.
 +
* A downtime cannot be edited once it has finished, nor can a new downtime be added more than 48 hours into the past.
  
* [https://edms.cern.ch/file/829986/0.1/EGEE-downtime-notification-procedure.pdf View documentation about Scheduled Downtime notification procedure]
+
If for any reason a downtime already declared needs to be extended, the procedure is to add another adjacent downtime, before or after.
  
 +
= Service types =
 +
In GOCDB a service type is a technology used to provide a service. Each service endpoint in GOCDB is associated with a service type. Service types are pieces of software while service endpoints are a particular instance of that software running in a certain context.
  
=== Downtime shortening and extension ===
+
===Service Type Naming Scheme===
 +
* Service types include grid middleware and operational services.
 +
* This attribute corresponds to the Glue2 'Service.Type' attribute and is defined as the 'Type of service according to a namespace based classification (the namespace MAY be related to a middleware name, an organisation or other concepts).'
 +
* The naming scheme for <i>new</i> service types in GOCDB therefore generally follow a reverse DNS style syntax, usually naming the technology provider/project followed by technology type in lowercase, i.e. ‘<provider>.<type>’ (e.g. ‘org.openstack.swift’).
 +
* Please note, this syntax does not necessarily indicate ownership, the main objective is to avoid name clashes between services. For example, different projects may have similar services but these may be modified/customised just enough to merit a different prefix or service type name. 
 +
* Glue2 defines a service type list at: [[https://github.com/OGF-GLUE/Enumerations Glue2 Enums]] [[https://github.com/OGF-GLUE/Enumerations/blob/master/ServiceType_t.csv Glue2 service types]].
 +
* The Glue2 and GOCDB recommendation is to use lowercase (legacy enum values do exist that use camelCase).
 +
<!--[[https://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/ServiceTypes GLUE2 service types]].--!> 
 +
* It would be preferable to rename all existing and legacy service types using the GLUE2 values, but this is potentially problematic for existing services that depend on established legacy names.
 +
* Service types with the 'egi.' or 'ngi.' are used to distinguish between different functional implementations of a service (i.e. variations of a service type with different functions).
  
Limition rules to downtime extensions are enforced in GOCDB as follows:
+
====Service types without a CUSTOM prefix====
* Once created, downtimes can be shortened but not extended
+
The software is recognized as a reusable product. Its scope for use is beyond that of a single organisational unit (VO/NGI). Its service type name should reflect a particular technology provider/product, not a generic term such as ‘Helpdesk’.
* If for any reason a downtime already declared needs to be extended, the procedure is to add another adjencent downtime, before or after.
+
* A service type should name a technology/product that is used to provide a service
* Any downtime can be shortened to any date which is not in the past.
+
* The name of a service type should begin with a reverse DNS representation of the technology provider (e.g. eu.eu-emi.Technology)
* A downtime's start and end time cannot be changed once the downtime has finished
+
* The name of the techology should appear after the technology provider e.g. org.irods.irods3
 +
* The name should reflect a particular technology/product, not a generic term such as Helpdesk
 +
* The service type name should not indicate the scope in which a technology is used
  
= Service types =
+
====Service types with a CUSTOM prefix====
 +
Software which is not a generically reusable product: This includes software whose scope may be limited to a specific organizational unit (e.g. NGI or VO), or software which has been customised so that its functionality is no longer standard. The service type name may reflect the usable scope of deployment (e.g. naming the OU). Note, with this definition, CUSTOM does not reflect extend/popularity of service deployment.
 +
<!--Service types prefixed with CUSTOM are used within a limited scope (e.g. by a single NGI, site or VO). -->
 +
These service types are used at some grid sites within EGI but aren't EGI operational tools or a part of the core middleware distributions (EMI, gLite, ARC, UNICORE, Globus etc).
  
Service types include grid middleware and operational services. The naming scheme for <i>new</i> service types follow a reverse DNS style syntax, usually naming the technology provider followed by technology type, i.e. ‘<provider>.<type>’ (e.g. ‘unicore6.StorageFactory’). This style is consistent with GLUE2 which also defines a service type list [[https://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/ServiceTypes GLUE2 service types]]. It would be preferable to rename all existing service types using the GLUE2 values, but this is potentially problematic for existing services that depend on established legacy names.
+
====Service Type List====
Service types with the 'egi.' or 'ngi.' prefixes do not refer to scope, but instead are intended to distinguish between different functional implementations of a service (i.e. variations of a service type with different functions).
+
To request a new service type, please submit a request for a new service type (described below).
  
To request a new servcie type, please submit a request for a new service type (described below).
+
<b>Operational Components (middleware agnostic)</b><br/>
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=Site-BDII Site-BDII]''':  [Site service] ''This service collects and publishes site's data for the Information System. All grid sites MUST install one Site-BDII. For cloud sites eu.egi.cloud.information.bdii MUST be installed.''
The current list consists of operational components and middleware:
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=Top-BDII Top-BDII]''':  [Central service] ''The "top-level BDII". These collect and publish the data from site-BDIIs. Only a few instances per region are required. ''
 
 
===Operational Components (middleware agnostic)===
 
* '''Site-BDII''':  [Site service] ''This service collects and publishes site's data for the Information System. All sites MUST install one Site-BDII. ''
 
* '''Top-BDII''':  [Central service] ''The "top-level BDII". These collect and publish the data from site-BDIIs. Only a few instances per region are required. ''
 
* '''OpsTool''': [Central service] generic service representing an operation tool (topology repository, dashboard, helpdesk system...)
 
* <strike>'''RGMA-IC''':  [OBSOLETE Central service]  ''A Registry for an R-GMA service. There will only ever be a few of these per grid. ''</strike>
 
 
<!--* '''Site-NAGIOS''': [Site service] site-level Nagios monitoring box-->
 
<!--* '''Site-NAGIOS''': [Site service] site-level Nagios monitoring box-->
 
<!--* '''Regional-NAGIOS''': [Regional Service] ROC-level Nagios monitoring box-->
 
<!--* '''Regional-NAGIOS''': [Regional Service] ROC-level Nagios monitoring box-->
 
<!--* '''Project-NAGIOS''': [Central Service] project-level Nagios monitoring box-->
 
<!--* '''Project-NAGIOS''': [Central Service] project-level Nagios monitoring box-->
* '''MyProxy''':  [Central service] MyProxy is part of the authentication and authorization system. Often installed by sites installing the WMS service.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=MyProxy MyProxy]''':  [Central service] MyProxy is part of the authentication and authorization system. Often installed by sites installing the WMS service.
* '''egi.APELRepository''': [Central service] The central APEL repository
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.APELRepository egi.APELRepository]''': [Central service] The central APEL repository
* '''egi.AccountingPortal''': [Central service] The central accounting portal
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.AccountingPortal egi.AccountingPortal]''': [Central service] The central accounting portal
* '''egi.GGUS''': [Central service] The central GGUS
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.GGUS egi.GGUS]''': [Central service] The central GGUS
* '''egi.GOCDB''': [Central service] The central GOCDB
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.GOCDB egi.GOCDB]''': [Central service] The central GOCDB
* '''egi.MSGBroker''': [Central service] The central message broker
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.MSGBroker egi.MSGBroker]''': [Central service] The central message broker
* '''MSG-Broker''': [Central service] A broker for the backbone messaging system.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.Portal egi.Portal]''': [Central Service] for monitoring generic web portals who dont have a specific service type
* '''egi.MetricsPortal''': [Central service] The central metrics portal
+
* <strike>'''MSG-Broker''': [Central service] A broker for the backbone messaging system.</strike>
* '''egi.NetworkPortal''': [Central service] The central network portal
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.MetricsPortal egi.MetricsPortal]''': [Central service] The central metrics portal
* '''egi.OpsPortal''': [Central service] The central operations portal
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.OpsPortal egi.OpsPortal]''': [Central service] The central operations portal
* '''egi.GRIDVIEW''': [Central service] The central gridview portal
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.GRIDVIEW egi.GRIDVIEW]''': [Central service] The central gridview portal
* '''egi.GSTAT''': [Central service] The central GStat portal
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.GSTAT egi.GSTAT]''': [Central service] The central GStat portal
* '''egi.SAM''': [Central service] The central SAM monitoring
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.SAM egi.SAM]''': [Central service] The central SAM monitoring
* '''ngi.SAM''': [Regional Service] NGI-level SAM monitoring box
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=ngi.SAM ngi.SAM]''': [Regional Service] NGI-level SAM monitoring box
* '''vo.SAM''': [Regional Service] VO-level SAM monitoring box
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=vo.SAM vo.SAM]''': [Regional Service] VO-level SAM monitoring box
* '''site.SAM''': [Regional Service] Site-level SAM monitoring box
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=site.SAM site.SAM]''': [Regional Service] Site-level SAM monitoring box
* '''ngi.OpsPortal''': [Regional service] NGI-level regional operations portal instance
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=ngi.OpsPortal ngi.OpsPortal]''': [Regional service] NGI-level regional operations portal instance
 +
<!--
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.egrant egi.egrant]'''  Platform for Resource Allocation management in EGI Infrastructure.  https://e-grant.egi.eu
 +
-->
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.MPI eu.egi.MPI]''': Defines a dummy Service Type to enable the running of MPI tests for services providing MPI capabilities. Sites must have one instance of this Service Type associated with a CREAM-CE service. For details see https://wiki.egi.eu/wiki/VT_MPI_within_EGI:Nagios
 +
<!--Defines an MPI test probe that is independent of a grid information system. This service will allow testing of sites which are offering MPI functionality but are not broadcasting it, or sites which are broadcasting the MPI/Parallel support in an incorrect way, see https://wiki.egi.eu/wiki/VT_MPI_within_EGI:Nagios
 +
-->
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=argo.poem argo.poem]''': POEM is system for managing profiles of probes and metrics in ARGO system.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=argo.mon argo.mon]''': ARGO Monitoring Engine gathers monitoring metrics and publishes to messaging service.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=argo.consumer argo.consumer]''': ARGO Consumer collects monitoring metrics from monitoring engines.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=argo.computeengine argo.computeengine]''': ARGO Compute Engine computes availability and reliability of services.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=argo.api argo.api]''': ARGO API service for retrieving status and A/R results.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=argo.webui argo.webui]''': ARGO web user interface for metric A/R visualization and recalculation management.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.aai.saml egi.aai.saml]''': EGI AAI CheckIn SAML interface. Enables federated access to EGI services and resources using Security Assertion Markup Language (SAML). Provided by GRNET.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.aai.oidc egi.aai.oidc]''': EGI AAI CheckIn OpenID Connect interface. Enables federated access to EGI services and resources using OpenID Connect (OIDC). Provided by GRNET.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=egi.aai.tts egi.aai.tts]''': EGI AAI CheckIn token translation service. Enables the translation between different authentication and authorisation protocols. Provided by GRNET.
  
===EMI Middleware (ARC, gLite, Unicore)===
 
=====ARC Middleware=====
 
* '''ARC-CE''':  [Site service] ''The Compute Element within the ARC middleware stack. ''
 
* '''SGAS''': [Site service] An accounting service used by ARC.
 
  
=====gLite Middleware=====
+
<b>Middleware (ARC, gLite, Unicore)</b><br/>
* '''CE''':  [Site service] ''The LCG Compute Element. Currently the standard CE within the gLite middleware stack. Soon to be replaced by the CREAM CE. ''
+
<b>ARC Middleware</b><br/>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=ARC-CE ARC-CE]''':  [Site service] The Compute Element within the ARC middleware stack.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=SGAS SGAS]''': [Site service] An accounting service used by ARC.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.nordugrid.arex org.nordugrid.arex]''': [Site Service] ARC version 3 Compute element. 
 +
 
 +
<b>gLite Middleware</b><br/>
 +
* <strike>'''CE''':  [OBSOLETE Site service] ''The LCG Compute Element. Currently the standard CE within the gLite middleware stack.</strike> Replaced by the CREAM CE. ''
 
* <strike>'''gLite-CE''':  [OBSOLETE Site service] ''The gLite Compute Element is now obsolete and is not supported. Please avoid using this middleware service.</strike>''
 
* <strike>'''gLite-CE''':  [OBSOLETE Site service] ''The gLite Compute Element is now obsolete and is not supported. Please avoid using this middleware service.</strike>''
* '''CREAM-CE''':  [Site service] ''The CREAM Compute Element is the new CE within the gLite middleware stack. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CREAM-CE CREAM-CE]''':  [Site service] ''The CREAM Compute Element is the new CE within the gLite middleware stack. ''
* '''APEL''':  [Site service] ''This is a "dummy" Service Type to enable the monitoring tests for APEL accounting. All sites must have one instance of this Service Type, associated with a CE. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=APEL APEL]''':  [Site service] ''This is a "dummy" Service Type to enable the monitoring tests for APEL accounting. All sites must have one instance of this Service Type, associated with a CE. ''
 
* <strike>'''MON''':  [OBSOLETE Site service] ''The gLite MonBox hosts the site R-GMA services.</strike>''
 
* <strike>'''MON''':  [OBSOLETE Site service] ''The gLite MonBox hosts the site R-GMA services.</strike>''
* '''UI''':  [User service]  ''The User Interface. Can be installed by users but more commonly installed by a site. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=UI UI]''':  [User service]  ''The User Interface. Can be installed by users but more commonly installed by a site. ''
* '''SRM''':  [Site service] ''Storage Resource Manager. Mandatory for all sites running an SRM enabled storage element. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=SRM SRM]''':  [Site service] ''Storage Resource Manager. Mandatory for all sites running an SRM enabled storage element. ''
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=SRM.nearline SRM.nearline]''':  [Site service] ''Storage Resource Manager for tape only.''
 +
<strike>* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=SRM.online SRM.online]''':  [Site service] ''Storage Resource Manager for disk only.''</strike>
 
* <strike>'''Classic-SE''':  [OBSOLETE Site service] ''The Classic Storage Element is now obsolete and is not supported. Please avoid using this middleware service.</strike>''
 
* <strike>'''Classic-SE''':  [OBSOLETE Site service] ''The Classic Storage Element is now obsolete and is not supported. Please avoid using this middleware service.</strike>''
* '''Central-LFC''':  [Central service] ''An instance of the gLite file catalogue which holds entries for all files owned by a particular VO. NOTE: An LFC can be both Central and Local. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=Central-LFC Central-LFC]''':  [Central service] ''An instance of the gLite file catalogue which holds entries for all files owned by a particular VO. NOTE: An LFC can be both Central and Local. ''
* '''Local-LFC''':  [Site service] ''An instance of the gLite file catalogue which holds entries for files owned by a particular VO, at your site. NOTE: An LFC can be both Central and Local. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=Local-LFC Local-LFC]''':  [Site service] ''An instance of the gLite file catalogue which holds entries for files owned by a particular VO, at your site. NOTE: An LFC can be both Central and Local. ''
* '''WMS''':  [Central service]  ''gLite Workload Management Service. Acts as the broker for matching user jobs to available computing resources. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=WMS WMS]''':  [Central service]  ''gLite Workload Management Service. Acts as the broker for matching user jobs to available computing resources. ''
 
* <strike>'''RB''':  [OBSOLETE Central service]  ''The LCG Resource Broker is now obsolete and is not supported. Please avoid using this middleware service.</strike>''
 
* <strike>'''RB''':  [OBSOLETE Central service]  ''The LCG Resource Broker is now obsolete and is not supported. Please avoid using this middleware service.</strike>''
* '''VOMS''':  [Central service] '' VO Management System. Part of the authentication and authorization system. This service only needs to be installed on the request of a VO. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=VOMS VOMS]''':  [Central service] '' VO Management System. Part of the authentication and authorization system. This service only needs to be installed on the request of a VO. ''
* '''LB''':  [Central service] '' gLite Logging and Bookkeeping. Usually installed by sites running a WMS. One LB service can support several WMS instances. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=LB LB]''':  [Central service] '' gLite Logging and Bookkeeping. Usually installed by sites running a WMS. One LB service can support several WMS instances. ''
* '''AMGA''':  [Central service]  ''gLite metadata catalogue. This service only needs to be installed on the request of a VO. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=AMGA AMGA]''':  [Central service]  ''gLite metadata catalogue. This service only needs to be installed on the request of a VO. ''
* '''FTM''': [Site service]  ''gLite File Transfer Monitor. Monitors the FTS service at a site. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=FTM FTM]''': [Site service]  ''gLite File Transfer Monitor. Monitors the FTS service at a site. ''
* '''FTS''':  [Central service]  ''The gLite File Transfer Service manages the transfer of files between sites. This service only needs to be installed on the request of a VO. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=FTS FTS]''':  [Central service]  ''The gLite File Transfer Service manages the transfer of files between sites. This service only needs to be installed on the request of a VO. ''
* '''VO-box''':  [Site service]  ''The gLite VO box allows a VO to run their own services at a site. This service only needs to be installed on the request of a VO. ''
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=VO-box VO-box]''':  [Site service]  ''The gLite VO box allows a VO to run their own services at a site. This service only needs to be installed on the request of a VO. ''
* '''gLite-APEL''': [Site service] The gLite-APEL hosts the site Accounting client (3.2 replacement of the MonBox)
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=gLite-APEL gLite-APEL]''': [Site service] The gLite-APEL hosts the site Accounting client (3.2 replacement of the MonBox)
* '''gLExec''': [Site service] A light-weight gatekeeper to authenticate and authorize credentials according to local site policy and execute commands. https://www.nikhef.nl/pub/projects/grid/gridwiki/index.php/GLExec
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=gLExec gLExec]''': [Site service] A light-weight gatekeeper to authenticate and authorize credentials according to local site policy and execute commands. https://www.nikhef.nl/pub/projects/grid/gridwiki/index.php/GLExec
* '''emi.ARGUS''':  [Site service] The Argus Authorization Service renders XACML authorization decisions for distributed services, based on policies
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=emi.ARGUS emi.ARGUS]''':  [Site service] The Argus Authorization Service renders XACML authorization decisions for distributed services, based on policies
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=ngi.ARGUS ngi.ARGUS]''': [Central Service] Distinguishes site Argus instances (emi.ARGUS) from NGI ones (ngi.ARGUS). While servicegroups can be used for this in gocdb, this is needed for backward-compatibility with the monitoring.
 +
 
 +
<b>Unicore Middleware</b><br/>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.unicore.UnicorePortal eu.unicore.UnicorePortal]''' UNICORE Portal service allows users to access a UNICORE Grid via the web browser. Service be used by users instead of standalone clients like UCC and URC to access resources and submit jobs. The UNICORE Portal server is available for download at UNICORE site: http://unicore.eu/download/unicore6/. It is possible in the future that each NGI supporting UNICORE middleware will deploy at least one instance.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.Registry unicore6.Registry]''': [Central service] All UNICORE services register here; clients ask the registry for available services in the Grid. Normally one Registry per Grid infrastructure which collects URLs of services.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.Gateway unicore6.Gateway]''': [Site service] Sits in front of one or more UNICORE services as a gateway to the internet. Normally one Gateway per site.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.TargetSystemFactory unicore6.TargetSystemFactory]''' [Site service] used as an entry-point for submitting single jobs. It can create Target System Services (TSSs) and submit jobs to those TSSs.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.StorageFactory unicore6.StorageFactory]'''  [Site service] Creates StorageManagement instances. A user can create dynamic storage management services for own purposes with it. Often used to provide filespace during workflow execution.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.StorageManagement unicore6.StorageManagement]'''  [Site service] Provides an abstract filesystem-like view on a storage resource. A Storage Management Service (SMS) can be created by a Storage Factory or can be configured statically way by a config file.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.ServiceOrchestrator unicore6.ServiceOrchestrator]'''  [Site service] Handles dispatching of a workflow's atomic jobs, and brokering. Normally there is one per grid infrastructure.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.WorkflowFactory unicore6.WorkflowFactory]'''  [Site service] Used as an entrypoint for submitting workflow jobs. The Workflow factory is creating workflow instances and can submit workflows to them. It is the workflow submission equivalent to the Target System Factory used for single job submission.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=unicore6.UVOSAssertionQueryService unicore6.UVOSAssertionQueryService]'''  [Site service] Provides data and user information via the SAML standard as needed for authorization and environment customization.
 +
 
 +
<b>Globus Middleware</b><br/>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=GRAM5 GRAM5]''': [Site service] job submission service for Globus version 5.x (GRAM5).
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=globus-GRIDFTP globus-GRIDFTP]''': [Site service] storage endpoint and data transfer service for the Globus middleware stack.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=globus-GSISSHD globus-GSISSHD]''': [Site service] certificate based interactive login service for the Globus middleware stack.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=MyProxy MyProxy]''': [Site service] MyProxy is part of the authentication and authorization system.
 +
 
 +
<b>QosCosGrid (QCG) Middleware</b><br/>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=QCG.Computing QCG.Computing]''' [Site service] A compute component based on the OGF Basic Execution Service (BES) with advanced reservation support.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=QCG.Notification QCG.Notification]''' [Site service] A notification middleware component using a brokered version of the OASIS WS-Notification standard.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=QCG.Broker QCG.Broker]''' [Site service] QosCosGrid resource management and brokering service.
 +
 
 +
<b>EDGI Middleware (European Desktop Grid Initiative)</b><br/>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=dg.CREAM-CE dg.CREAM-CE]''' CREAM gateway to Desktop Grid 
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=dg.3GBridge dg.3GBridge]''' The 3G Bridge (Generic Grid Grid Bridge) is designed to be used as a mediator between different types of Grid middleware.
  
=====Unicore Middleware=====
+
<b>Cloud</b><br/>
* '''unicore6.Registry''': [Central service] All UNICORE services register here; clients ask the registry for available services in the Grid. Normally one Registry per Grid infrastructure which collects URLs of services.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.accounting eu.egi.cloud.accounting]''' This Service Type is for Cloud accounting. All sites which are Federated Cloud Resource Providers must have one instance of this Service Type
* '''unicore6.Gateway''': [Site service] Sits in front of one or more UNICORE services as a gateway to the internet. Normally one Gateway per site.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.vm-management.occi eu.egi.cloud.vm-management.occi]''' EGI cloud virtual machine management based on OCCI 1.1 specification http://occi-wg.org
* '''unicore6.TargetSystemFactory''' [Site service] used as an entry-point for submitting single jobs. It can create Target System Services (TSSs) and submit jobs to those TSSs.  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.storage-management.cdmi eu.egi.cloud.storage-management.cdmi]''' EGI cloud data management interface based on CDMI 1.0.2 specification http://www.snia.org/cdmi
* '''unicore6.StorageFactory''' [Site service] Creates StorageManagement instances. A user can create dynamic storage management services for own purposes with it. Often used to provide filespace during workflow execution.  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.vm-metadata.marketplace eu.egi.cloud.vm-metadata.marketplace]''' EGI cloud virtual machine image metadata repository
* '''unicore6.StorageManagement''' [Site service] Provides an abstract filesystem-like view on a storage resource. A Storage Management Service (SMS) can be created by a Storage Factory or can be configured statically way by a config file.  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.information.bdii eu.egi.cloud.information.bdii]''' FedCloud specific LDAP server which speaks GLUE2
* '''unicore6.ServiceOrchestrator''' [Site service] Handles dispatching of a workflow's atomic jobs, and brokering. Normally there is one per grid infrastructure.  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.broker.compss eu.egi.cloud.broker.compss]''' A super-scalar broker service and programming model for Grids and Clouds. From Barcelona Supercomputing Centre (BSC). <!--daniele.lezzi@bsc.es-->
* '''unicore6.WorkflowFactory''' [Site service] Used as an entrypoint for submitting workflow jobs. The Workflow factory is creating workflow instances and can submit workflows to them. It is the workflow submission equivalent to the Target System Factory used for single job submission.  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.broker.proprietary.slipstream eu.egi.cloud.broker.proprietary.slipstream]''' A broker supporting autoscaling and elasticity of virtualised applications in federated Clouds. From SixSq AG. <!--meb@sixsq.com -->
* '''unicore6.UVOSAssertionQueryService''' [Site service] Provides data and user information via the SAML standard as needed for authorization and environment customization.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.broker.vmdirac eu.egi.cloud.broker.vmdirac]''' A IaaS cloud broker extending Dirac to automated VM based application deployment. From Universitat Autònoma de Barcelona (UAB). <!-- vmendez@aomail.uab.es-->
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.vm-metadata.vmcatcher eu.egi.cloud.vm-metadata.vmcatcher]''' VMCatcher deployment on fedcloud sites
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.egi.cloud.vm-metadata.appdb-vmcaster eu.egi.cloud.vm-metadata.vmcatcher]''' A service responsible for managing and publishing registered images to the EGI AppDB Cloud marketplace, following the HEPiX image list format.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.openstack.nova org.openstack.nova]''' OpenStack Nova provides VM management services.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.openstack.swift org.openstack.swift]''' OpenStack Swift is a multi-tenant object storage system that manages unstructured data through a RESTful HTTP API.
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=com.ceph.object-storage com.ceph.object-storage]''' Object Store implementation from Ceph, see: http://ceph.com/ceph-storage/object-storage
  
===Globus Middleware===
+
<b>Other</b><br/>
* '''GRAM5''': [Site service] job submission service for Globus version 5.x (GRAM5).
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=eu.unity-idm.Unity eu.unity-idm.Unity]''' Unity is a complete solution for identity, federation and inter-federation management. Or, looking from a different perspective, it is an extremely flexible authentication service. (http://www.unity-idm.eu). It replaces UVOS service for UNICORE. Possibly each NGI supporting UNICORE should deploy one instance.
* '''globus-GRIDFTP''': [Site service] storage endpoint and data transfer service for the Globus middleware stack.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=com.schedmd.SlurmClient com.schedmd.SlurmClient]''' SLURM (Simple Linux Utility for Resource Management) is an open-source resource manager designed for Linux clusters of all sizes. Machine flagged this type is hosting software to submit jobs to SLURM. http://slurm.schedmd.com/
* '''globus-GSISSHD''': [Site service] certificate based interactive login service for the Globus middleware stack.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.ogf.bes.BESFactory org.ogf.bes.BESFactory]''' OGSA BES Factory implementations.
* '''MyProxy''': [Site service] MyProxy is part of the authentication and authorization system.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.squid-cache.Squid org.squid-cache.Squid]''' The Squid web proxy service. htp;//www.squid-cache.org. Original request: https://rt.egi.eu/rt/Ticket/Display.html?id=3482
* '''globus-RLS''': [Site service] The globus Replica Location Service.
 
  
===QosCosGrid (QCG) Middleware===
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.irods.irods3 org.irods.irods3]''' An Integrated Rule-Oriented Data System for grids, version 3.  
* '''QCG.Computing''' [Site service] A compute component based on the OGF Basic Execution Service (BES) with advanced reservation support.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.irods.irods4 org.irods.irods4]''' An Integrated Rule-Oriented Data System for grids, version 4.
* '''QCG.Notification''' [Site service] A notification middleware component using a brokered version of the OASIS WS-Notification standard.
 
* '''QCG.Broker''' [Site service] QosCosGrid resource management and brokering service.
 
  
=== EDGI Middleware (European Desktop Grid Initiative) ===
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.irods.irods3.icat org.irods.irods3.icat]''' An Integrated Rule-Oriented Data System for grids, icat server version 3.
* '''dg.CREAM-CE''' CREAM gateway to Desktop Grid
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.irods.irods4.icat org.irods.irods4.icat]''' An Integrated Rule-Oriented Data System for grids, icat server version 4.
* '''dg.ARC-CE''' ARC gateway to Desktop Grid
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.irods.irods3.resource org.irods.irods3.resource]''' An Integrated Rule-Oriented Data System for grids, resource server package version 3.
* '''dg.TargetSystemFactory''' UNICORE gateway to Desktop Grid
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.irods.irods4.resource org.irods.irods4.resource]''' An Integrated Rule-Oriented Data System for grids, resource server package version 4. 
  
=== iRods===
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=pl.cyfronet.gridspace2 pl.cyfronet.gridspace2]''' Virtual laboratory framework enabling researchers to conduct virtual experiments on Grid-based resources. http://dice.cyfronet.pl/gridspace
* '''org.irods.irods3''' An Integrated Rule-Oriented Data System for grids.  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=pl.psnc.MigratingDesktop pl.psnc.MigratingDesktop]''' Client framework for accessing the Grid resources. http://desktop.psnc.pl
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=pl.cyfronet.InSilicoLab pl.cyfronet.InSilicoLab]''' InSilicoLab portal instance. http://insilicolab.grid.cyfronet.pl
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.vinetoolkit.VinePortal org.vinetoolkit.VinePortal]''' Vine toolkit portal instance. http://vinetoolkit.org
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=com.adaptivecomputing.TorqueClient com.adaptivecomputing.TorqueClient]''' TORQUE (Tera-scale Open-source Resource and QUEue manager) is a resource manager providing control over batch jobs and distributed compute nodes. Torque is based on OpenPBS. http://www.adaptivecomputing.com/products/open-source/torque
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=net.perfSONAR.Bandwidth net.perfSONAR.Bandwidth]'''  Network performance monitoring tool for bandwith. http://www.perfsonar.net
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=net.perfSONAR.Latency net.perfSONAR.Latency]'''  Network performance monitoring tool for latency. http://www.perfsonar.net
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=XRootD XRootD]''' XrootD storage service implementation which supports xroot protocol  http://xrootd.slac.stanford.edu
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=XRootD.Redirector XRootD.Redirector]''' Data access redirection service using XRoot protocol
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=ch.cern.cvmfs.stratum.0 ch.cern.cvmfs.stratum.0]''' Service component (stratum.0) of CernVM file system http://cernvm.cern.ch/portal/filesystem
 +
<strike>* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=ch.cern.cvmfs.stratum.1 ch.cern.cvmfs.stratum.1]''' Service component (stratum.1) of CernVM file system http://cernvm.cern.ch/portal/filesystem</strike>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=uk.ac.gridpp.vac uk.ac.gridpp.vac]''' A virtual machine factory system for operating clusters at grid sites
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=hu.guse.ws-pgrade hu.guse.ws-pgrade]''' WS-PGRADE/gUSE-based science gateway deployment
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=org.opensciencegrid.htcondorce org.opensciencegrid.htcondorce]''' A special configuration of the HTCondor software designed to be a job gateway solution
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=uk.ac.gridpp.vcycle uk.ac.gridpp.vcycle]''' Vcycle manages the lifecycle of VMs running jobs on cloud resources. LHCb, ATLAS, CMS, and GridPP DIRAC VMs
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=webdav webdav]''' Service with extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers. <!-- https://ggus.eu/?mode=ticket_info&ticket_id=125988-->
  
===Custom Service Types===
+
<b>Custom Service Types</b><br/>
These service types are used at some grid sites within EGI but aren't a part of the core middleware distributions (EMI, gLite, ARC, UNICORE, Globus etc).
+
In order to control the proliferation of custom service types, please consider submitting a request for a new service type (described below) before using CUSTOM_SERVICE.
* '''CUSTOM.org.squid-cache.Squid''' The Squid web proxy service. htp;//www.squid-cache.org. Original request: https://rt.egi.eu/rt/Ticket/Display.html?id=3482
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.ch.cern.frontier.FroNTier CUSTOM.ch.cern.frontier.FroNTier]''' The Frontier system distributes data from central databases to many clients around the world. Used in ATLAS and CMS. http://frontier.cern.ch  
* '''CUSTOM.ch.cern.frontier.FroNTier''' The Frontier system distributes data from central databases to many clients around the world. Used in ATLAS and CMS. http://frontier.cern.ch  
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.UserPortal CUSTOM.UserPortal]''' A generic user portal for use by all NGIs
* '''CUSTOM_SERVICE''' Global catch-all type for custom or proprietary services that are not described above. <b>Important:</b> in order to control the proliferation of custom service types, please consider submitting a request for a new service type (described below) before using CUSTOM_SERVICE.
+
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.RequestTracker CUSTOM.RequestTracker]''' A generic request tracker for use by all NGIs
 +
<strike>* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.pl.plgrid.Bazaar CUSTOM.pl.plgrid.Bazaar]''' SLA negotiation system between users and resource providers from NGI_PL grid</strike>
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.pl.plgrid.BazaarSAT CUSTOM.pl.plgrid.BazaarSAT]''' Bazaar Site Admin Toolkit from NGI_PL grid
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.pl.plgrid.BAT.agent CUSTOM.pl.plgrid.BAT.agent]''' Service for collecting accounting data from NGI_PL grid
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.pl.plgrid.QStorMan.UserInterface CUSTOM.pl.plgrid.QStorMan.UserInterface]''' A service to provide a user of the grid system with a certain level of quality, from NGI_PL grid
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM.pl.plgrid.KeyFS CUSTOM.pl.plgrid.KeyFS]''' Key File System service, installed on UI machines to provide a user with the grid credentials, from NGI_PL grid
 +
* '''[https://goc.egi.eu/gocdbpi/public/?method=get_service_endpoint&service_type=CUSTOM_SERVICE CUSTOM_SERVICE]''' Global catch-all type for custom or proprietary services that are not described above.
  
 
==Adding new services types==
 
==Adding new services types==
''Please make a request to the EGI RT ticketing system if you would like a new service type to be registered in GOCDB. Please ensure that the ticket is submitted to the GOCDB Requirements queue in RT'' [https://rt.egi.eu/rt/Search/Results.html?Format=%27%20%20%20%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__id__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3A%23%27%2C%0A%27%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__Subject__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3ASubject%27%2C%0A%27__Status__%27%2C%0A%27__QueueName__%27%2C%0A%27__OwnerName__%27%2C%0A%27__Priority__%27%2C%0A%27__NEWLINE__%27%2C%0A%27%27%2C%0A%27%3Csmall%3E__Requestors__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__CreatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__ToldRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__LastUpdatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__TimeLeft__%3C%2Fsmall%3E%27&Order=ASC|ASC|ASC|ASC&OrderBy=id|||&Query=Queue%20%3D%20%27requirements%27%20AND%20%28%20%20Status%20%3D%20%27new%27%20OR%20Status%20%3D%20%27open%27%20OR%20Status%20%3D%20%27accepted%27%20OR%20Status%20%3D%20%27developed%27%20OR%20Status%20%3D%20%27stalled%27%20OR%20Status%20%3D%20%27feedback%27%20%29%20AND%20%27CF.{Category%20%28level%202%29}%27%20LIKE%20%27GOCDB%20%28Operational%20Tools%29%27%20AND%20Queue%20%3D%20%27requirements%27%20AND%20%27CF.{Category%20%28level%201%29}%27%20LIKE%20%27Operational%20Tools%27&RowsPerPage=50&SavedChartSearchId=new&SavedSearchId=RT%3A%3AUser-3851-SavedSearch-1174 GOCDB RT Requirements Queue Tickets]
+
Please feel free to make a request for a new service type. For CUSTOM service types, we would like to make this process as light-weight as possible. However, currently all new service type requests need to be assessed by EGI via lightweight review process (by OMB and EGI Ops) so that only suitable types are added to GOCDB and to prevent duplication.
<br/><br/>
+
 
 +
You can submit your request via GGUS to the "Configuration and Topology Database (GOCDB)" support unit.
 +
 
 +
Please specify the following information as part of your request:
 +
<pre>
 +
- name of service type (lowercase):
 +
- high-level description of the service functionality (255 characters max):
 +
- project/community/organization maintaining the software:
 +
- scale of deployment (number of instances and by which organizations):
 +
- contact point (name/e-mail address):
 +
</pre>
 +
 
 +
Note - please provide a suggested SE type name following the naming scheme described above (technology provider's reversed domain . software name) and a brief sentence to describe the service type.
 +
 
 +
<s>Guidelines [https://wiki.egi.eu/wiki/Adding_Custom_Service_to_Availability_Monitoring here] for adding custom service types to SAM for monitoring.</s>
 +
 
 +
= Data Visibility / Scopes =
 +
* Scope tags are used to group Grid entities such as Sites, Services and ServiceGroups into flexible categories. A single entity can define multiple scope tags, allowing the resource to be associated with different categories without duplication of information. This is essential to maintain the integrity of topology information across different infrastructures and projects.  
 +
* The GOCDB  admins  control  which  scope  tags  are  made  available  to  avoid  proliferation  of  tags  (user defined tags are reserved for the extensibility mechanism).  
 +
* As an example, a site’s scope list could aggregate all of the scopes defined by its child services. In doing this, the site scope list becomes a union of its service scopes plus any other site specific tags defined by the site.  
 +
* By defining scope tags, resources can be ‘filtered-by-scope-tag’ when querying for data in the PI using the ‘scope’ and ‘scope_match’ parameters, see [[GOCDB/PI/Technical_Documentation | GOCDB Programmatic Interface (GOCDB-PI)]] for details.
 +
 
 +
== Clear Separation of Concerns ==
 +
It is important to understand that scopes and Projects are distinct:
 +
* Projects are used to cascade roles and permissions over child objects
 +
* Scope tags are used to filter resources into flexible categories/groupings
 +
* Scope tags can be created to mirror the projects. For example, assuming two projects (e.g. EGI.eu and EUDAT), two corresponding tags may be defined. 
 +
* In addition, it is also possible define additional scopes for finer grained resource filtering e.g. ‘SubGroupX’ and ‘EGI_TEST’.
 +
* The key benefit:  A clear separation of concerns between cascading permissions and resource filtering.
  
= Data Visibility =
+
== EGI Scopes ==
* If a site or service endpoint is marked as being visible to EGI then they will be exposed to the central operational tools. For example, marking a site as being visible to EGI will mean that it can be monitored centrally and it will appear in the central operations portal.
+
* To make a Site, Service or ServiceGroup visible to EGI, the resource's 'EGI' scope tag check box must be ticked. EGI scoped resources are exposed to the central operational tools for monitoring and will appear in the central operations portal.
  
* Un-ticking this box makes the selected object invisible to EGI; it will be hidden from the central operation tools (it will not show in the central dashboard and it will not be monitored centrally). This can be useful if you wish to hide certain parts of your infrastructure from EGI but still have the information stored and accessed from the same GOCDB instance.  
+
* <b>Un-ticking the EGI checkbox</b> and selecting the 'Local' scope makes the selected object invisible to EGI; it will be hidden from the central operation tools (it will not show in the central dashboard and it will not be monitored centrally). This can be useful if you wish to hide certain parts of your infrastructure from EGI but still have the information stored and accessed from the same GOCDB instance.  
  
* A use-case for non-EGI sites/SEs is to hide those entities from central EGI tools, but to include those sites/services for use by regional versions of the operational tools (such as regional monitoring). To enable regional monitoring of non-EGI sites/SEs using SAM see [[https://ggus.eu/ws/ticket_info.php?ticket=76888 original change request]] and [[https://tomtools.cern.ch/jira/browse/SAM-2285 Add support for GOCDB scope]]   
+
* A use-case for non-EGI sites/services is to hide those entities from central EGI tools, but to include those sites/services for use by regional versions of the operational tools (such as regional monitoring). To enable regional monitoring of non-EGI sites/services using SAM see [[https://ggus.eu/ws/ticket_info.php?ticket=76888 original change request]] and [[https://tomtools.cern.ch/jira/browse/SAM-2285 Add support for GOCDB scope]]   
  
 
* Note that exposing a site / service endpoint as EGI does not override the production status or certification status fields. For example if a site isn't marked as production it won't be monitored centrally even if it's marked as visible to EGI.
 
* Note that exposing a site / service endpoint as EGI does not override the production status or certification status fields. For example if a site isn't marked as production it won't be monitored centrally even if it's marked as visible to EGI.
  
= Groups =
+
* You can submit your request for new scope tags via GGUS to the "Configuration and Topology Database (GOCDB)" support unit.
A group is a grouping of sites in GOCDB. GOCDB stores the following information about groups:  
+
 
 +
==Reserved Scope Tags==
 +
* Some tags may be 'Reserved' which means they are protected - they are used to restrict tag usage and prevent non-authorised sites/services from using tags not intended for them.
 +
* Reserved tags are initially assigned to resources by the gocdb-admins, and can then be optionally inherited by child resources (tags can be initially assigned to NGIs, Sites, Services and ServiceGroups).
 +
* When creating a new child resource (e.g. a child Site or child Service), the scopes that are assigned to the parent are automatically inherited and assigned to the child.
 +
* Reserved tags assigned to a resource are optional and can be de-selected if required.
 +
* Users can reapply Reserved tags to a resource ONLY if the tag can be inherited from the parent Scoped Entity (parents include NGIs/Sites).
 +
** For Sites: If a Reserved tag is removed from a Site, then the same tag is also removed from all the child Services - a Service can't have a reserved tag that is not supported by its parent Site.
 +
** For NGIs: If a Reserved tag is removed from an NGI, then the same tag is NOT removed from all the child Sites - this is intentionally different from the Site->Service relationship.
 +
* To request a reserved scope tag, <b>an approval is required from the operators of the relevant resources</b>. Details on who to contact are listed below. Once authorisation is given, please contact the GOCDB admins with details of the approval (e.g. link to a GGUS ticket that approves the tag assignment).
 +
 
 +
=== FedCloud Reserved Tag=== 
 +
* Tag for resources that contribute to the EGI Federated Cloud. To request this tag, please contact the FedCloud operators / EGI Operations.
 +
 
 +
=== Elixir Reserved Tag ===
 +
* Tag for resources that contribute to the EGI Federated Cloud. To request this tag, please contact the operators of the ‘ELIXIR’ NGI in GOCDB.
 +
=== WLCG Reserved Tags===
 +
* A number of reserved scope tags have been defined for the WLCG: 
 +
** The ‘tierN’ tags should be requested for WLCG sites that are defined in REBUS (a management view of the WLCG infrastructure/sites). To request a ‘tierN’ tag, raise a ticket against the REBUS support unit in GGUS.
 +
** For the experiment VO tags (alice, atlas, cms, lhcb), raise a ticket with the relevant VO support unit.
 +
** The wlcg tag is a generic catch-all tag for sites/services with either tierN and VO tags and is used to gain an overall view of the WLCG infrastructure.
 +
===SLA Reserved Tag===
 +
* Entitities covered by an EGI VO SLA
 +
** This Tag will only be applied at the request of EGI operations
 +
 
 +
= Extension Properties =
 +
''NOTE: From GOCDB 5.7 (Autumn/Winter 2016) keys must be unique for a given site, service, or service endpoint, or service group.''
 +
* Sites, Services, service endpoints, and Service Groups can be extended by adding custom key-value pairs (this follows the GLUE2 extensibility mechanism).
 +
* Extension properties address a number of use cases, such as filtering Sites and/or Services that define particular properties, e.g. 'charging rate': https://rt.egi.eu/rt/Ticket/Display.html?id=3764
 +
* Selected methods in the GOCDB PI support the 'extensions' URL parameter. This parameter is used to filter resources according to the extensions they define (described below). 
 +
* Properties are rendered in the XML results of the Site/Service/ServiceGroup using the <EXTENSIONS> XML element, for an example see a [https://wiki.egi.eu/wiki/GOCDB/PI/get_service_endpoint_method sample output from get_service_endpoint]
 +
* Note, anyone with permissions over the target entity can add extension properties to that object. 
 +
* This allows 'Folksonomy' building: 'a user-generated system of classifying and organizing content into different categories by the use of metadata such as electronic tags'
 +
* A number of use cases can be addressed; e.g. filtering Sites that support a specific property, e.g. ‘P4U_Pilot_Cloud_Wall’
 +
* Key/value pairs (currently) prevent certain characters from being used in their values. This includes the equals and opening/closing parenthesis chars ‘=()’.  This is to simplify lexical parsing of the query. In addition, to guard against cross-site scripting attacks, the quote, double quote, semi-colon and back tick chars are also not allowed.   
 +
 
 +
== Extension Properties in the PI  ==
 +
 
 +
*Selected PI methods allow results to be filtered by extension properties via the 'extensions' PI parameter.
 +
*Supported methods include: get_site, get_site_list, get_service_endpoints and get_service_group, get_downtime, get_downtime_nested, get_site_list.
 +
*For individual method support please refer to the PI documentation: [[GOCDB/PI/Technical Documentation]]
 +
 
 +
*The format of the 'extensions' PI parameter is one or more (key=value) pairs enclosed in brackets.
 +
**The value part of a (k=v) pair can be ommitted if filtering by value is not required (i.e. '(somekey=)' means select all resources that define the 'somekey' property with any value.
 +
**(k=v) pairs can be optionally prefixed with one of following operators: AND, OR, NOT.
 +
**If no operator is specified before the FIRST (k=v) pair, then AND is assumed.
 +
**A single operator applies to ALL the (k=v) pairs to the right of the operator until another operator is encountered.
 +
**An AND forms a logical conjunction with any previously specified conditions.
 +
**An OR forms a logical disjunction with any previously specified conditions.
 +
**A NOT forms a logical conjunction with any previously specified conditions (it can be read as 'AND NOT')
 +
**Because an OR always forms a logical disjunction with any previously specified conditions, you can’t OR against a group occurring to the right that contains multiple k=v pairs e.g. the following is not supported (if there is sufficient demand, it could be considered for a future enhancement):  
 +
***((k=v1)AND(k=v2)) OR ((k=v3)AND(k=v4))
  
* A group name
+
<!--** A sequence of multiple operators of the same type form a logical conjunction, while a different type of operator forms a logical disjunction with any previously specified restrictions.-->
* A description of the group
 
* A group type (NGI, Country, Infrastructure...)
 
* An e-mail contact when relevant
 
  
 +
Examples:
  
== Viewing Groups and Group information ==
+
*Eg (note no leading AND):
 +
**(key1=val)(key2=va2)OR(key3=val3)(key4=val4)NOT(key5=val5)(key6=val6)&nbsp;&nbsp;&nbsp;is expanded to:
 +
**<font color="red">AND(key1=val)AND(key2=va2)</font><font color="green">OR(key3=val3)OR(key4=val4)</font><font color="blue">NOT(key5=val5)NOT(key6=val6)</font> &nbsp;&nbsp;&nbsp;which is interpreted as:
 +
**(((key1=val)AND(key2=va2))OR(key3=val3)) OR(key4=val4) NOT(key5=val5) NOT(key6=val6)
  
The main page listing groups actually shows NGIs/ROCs, and is available from
+
*Eg:
* '''List of NGIs/ROCs and associated contacts''', linked from the main menu
+
**(VObing=true)AND(VObaz=true)AND(VObar=true)OR(s1p1=v1)&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;is equal to:
 +
**is equal to: <font color="red">(</font>(VObing=true)AND(VObaz=true)AND(VObar=true)<font color="red">)</font>OR(s1p1=v1)
  
Groups are also listed from site details pages (all groups the site belongs to). Because groups are generic entities in GOCDB4, there are many logical notions that are presented in this way: NGIs, Countries, Production Infrastructure... pretty much everything that groups sites together is defined as a group.
+
*Eg:
 +
**(VO=food)OR(VO2=bar)AND(s4p1=v1) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;is equal to:
 +
**<font color="red">(</font>(VO=food)OR(VO2=bar)<font color="red">)</font>AND(s4p1=v1)
  
Each group has its own listing page, accessible by clicking on the "view" link in group listing pages. A group details page shows users with a role on that group, as well as member sites and associated contacts and roles.
+
*Eg:
 +
**(VO=food)(s4p1=v1)OR(VObar=true)(VObaz=true) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;is equal to:
 +
**is equal to: <font color="red">(</font>(VO=food)AND(s4p1=v1)<font color="red">)</font>OR(VObar=true)OR(VObaz=true)
  
== Adding Groups ==
+
*Eg:
 +
**(VO=food)(s4p1=v1)OR(VObaz=true)AND(VObling=true) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;is equal to:
 +
**<font color="red">(</font><font color="blue">(</font>(VO=food)AND(s4p1=v1)<font color="blue">)</font>OR(VObaz=true)<font color="red">)</font>AND(VObling=true)
  
Adding groups is not possible through the Input System web interface.
+
<br> To return all sites that define VO with a value of Alice:
If you want to start the registration process of a new NGI, please follow the procedure described on:
+
<pre>?method=get_site&amp;extensions=(VO=Alice)</pre>
* [[Operations Centre creation process coordination]]
+
Use no value to define a wildcard search, i.e. all sites that define the VO property regardless of value:  
 +
<pre>?method=get_site&amp;extensions=(VO=)</pre>
 +
''<font color="red">NOTE: From GOCDB 5.7 (Autumn/Winter 2016) keys must be unique for a given site, service, or service endpoint, or service group. The following section of documentation has not yet been changed to reflect this.</font>''
  
Integration of the new group in GOCDB is part of the procedure but has to be done by GOCDB admins.
+
Extensions also supports OR/AND/NOT operators. This can be used to search against multiple key values eg:
 +
<pre>?method=get_site&amp;extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)</pre>
 +
These can be used together:
 +
<pre>?method=get_site&amp;extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)</pre> <pre>?method= get_service_endpoint&amp;extensions=(CPU_HS01_HOUR=1)OR(CPU_HS02_HOUR=2)</pre>
 +
When no operator is specified the default is AND, therefore the following:
 +
<pre>?method= get_service_endpoint&amp;extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)</pre>
 +
Is the same as:
 +
<pre>?method= get_service_endpoint&amp;extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)</pre>
 +
The extensions parameter can also be used in conjunction with the existing parameters previously supported:
 +
<pre>?method=get_site&amp;extensions=(VO=Alice)NOT(VO=LHCB)&amp;scope=EGI&amp;roc=NGI_UK</pre>
 +
*The 'site_extensions' and 'service_extensions' can also be used on the 'get_downtime' and 'get_downtime_nested_services' methods using same logic described above. Note, the &lt;EXTENSIONS&gt; element is not rendered in the XML output for these queries.
 +
<pre>?method=get_downtime_nested_services&amp;site_extensions=(eg.2=val.2)&amp;service_extensions=(eg.2=)</pre> <pre>?method=get_downtime&amp;site_extensions=(eg.2=val.2)&amp;service_extensions=(eg.2=)</pre>
  
== Editing Groups ==
+
== Standard Extension Properties ==
To edit a group, simply click on the "edit" link at the top of the group's details page
 
  
 +
=== HostDN ===
 +
''From GOCDB 5.8.0 onwards''
  
== Deleting Groups ==
+
For EGI Services, the Standard Extension property "'''HostDN'''" has been defined to allow the fetching the DNs of multiple hosts behind a load balanced service from the endpoint properties of a single GOCDB Service, rather than creating multiple GOCDB Services with different host DNs.
This operation is not allowed
 
  
<br/><br/>
+
==== Recommended Use ====
 +
To supply multiple or alternate DN(s) for a service, for example of the multiple hosts supporting a single service entry, the Service Extension Property (hereafter Ext) "HostDN" SHOULD be used. If Ext "HostDN" is present it MUST contain one or more x.509 DN values. Multiple values MUST be delimited by enclosing each within <> characters. If Ext "HostDN" is present, the Service "Host DN" SHOULD contain the x.509 SubjectAltName used in the x.509 certificate(s) presented by the hosts identified by the Ext "HostDN" values.
  
= How to and FAQ =
+
= Options for adding a new Project in GocDB =
 +
GocDB is multi-tenanted; it can host multiple projects in the same instance.  There are a number of different deployment scenarios that can be used to support new projects detailed below. Please contact the GocDB admins/EGI Ops to request the addition of a new project.
 +
=== 1) Add resources (sites/services) to an existing project ===
 +
* Resources (NGIs, Sites, Services) would be hosted under an existing project, e.g. the ‘EGI’ project.
 +
* The new resources would be subject to the rules/roles of the existing project, such as site certification status changes and project controlled user memberships.
 +
* The resources could not be filtered using a custom scope tag.
  
=== What is so different between GOCDB3 and GOCDB4? ===
+
=== 2) Add resources (sites/services) to an existing project and add a new Scope tag to represent a sub-grouping ===
 +
* Resources would be hosted under an existing project, and a new scope tag would be added for the purposes of resource filtering.
 +
* Since the resources are still hosted under an existing project, the resources would still be subject to the rules/roles of that project such as project controller user memberships.
 +
* The resources could be filtered using the new scope tag, but this scope tag would not strictly represent a project, rather a sub-grouping under the existing project, e.g. <pre>get_services&scope=SubGroupX</pre>  Note, resources can be tagged multiple times to declare support for multiple projects and sub-groups: <pre>get_services&scope=SubGroupX,EGI&scope_match=all</pre>
  
GOCDB3 and GOCDB4 are two animals with a similar shell but very different guts... you might want to browse the [[GOCDB/Release4/Architecture | GOCDB4 architecture documentation]] to know more about what makes GOCDB4 special.
+
=== 3) Add resources (sites/services) to a new Project and add a new Scope tag to filter by project ===
 +
* Resources would be hosted under a new project, and a new scope tag would be added named after the project for the purposes of resource filtering.
 +
* The resources would not be subject to the rules/roles of other projects, for example, allowing the project to control its own project memberships.  
 +
* The resources could be filtered using the scope tag named after the new project, e.g. <pre>get_services&scope=ProjectX</pre>  Note, resources can be tagged multiple times to declare support for multiple projects: <pre>get_services&scope=ProjectX,EGI&scope_match=all</pre>
 +
 
 +
= How to and FAQ =
  
 
=== I get an "error 12227" message when accessing GOC portal with Mozilla/Firefox ===
 
=== I get an "error 12227" message when accessing GOC portal with Mozilla/Firefox ===
Line 725: Line 1,352:
  
 
<br/><br/>
 
<br/><br/>
 
= Queries, contact and support =
 
== Send suggestions ==
 
 
Before you make any request, check this is not already integrated to our development plans. Any suggestion, new feature or improvement request should be submitted to our Savannah support tracker. Suggestions will be discussed within GOCDB developers, the OTAG, EGI Inspire-JRA1, or any political body involved before inclusion into development plans. These bodies reserve the right to decline unsuitable requests.
 
* check [[GOCDB/Documentation Index | GOCDB Documentation Index]] for up-to-date development plans
 
* [https://savannah.cern.ch/support/?func=additem&group=gocdb Access GOCDB Savannah support tracker]
 
 
 
== Report a bug ==
 
 
First,  check known bugs to see if this has not already been reported. If not, please create a new entry in our Savannah bug tracker, trying to be as precise and concise as possible.
 
* [https://savannah.cern.ch/bugs/?group=gocdb check known bugs]
 
* [https://savannah.cern.ch/bugs/?func=additem&group=gocdb Submit a bug in GOCDB bug tracker]
 
 
== Get some support ==
 
 
If you can't find what you are looking for in the documentation, as well as for all other enquiries including general questions, temporary problems reports or support requests, you can contact us using the mail below
 
 
* Contact GOCDB admins at [mailto:gocdb-admins_at_mailtalk.ac.uk gocdb-admins_at_mailtalk.ac.uk]
 

Latest revision as of 12:35, 13 August 2021

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


Contents

Introduction

Scope of this documentation

This user documentation is about the GOCDB5 Input System, which is either:

Other documentation

Version and improvements

This documentation is meant to be useful and accurate. If you think it is not, please send us any improvement suggestions to gocdb-admins_at_mailman.egi.eu

GOCDB version supported in this documentation: 5.3+

Quick Orientation guide

Accessing GOCDB5 input system

To access the web interface, you need an X509 digital certificate installed in your browser, delivered by one of the recognised EU-Grid-PMA Certification Authorities.

  • Obtain a X509 digital certificate
    • Please note, GOCDB does not support single or double quotes in the certificate DN (Distinguished Name).
      • This DN is rejected by GOCDB because of the single quote: /C=UK/O=STFC/OU=SomeOrgUnit/CN=David Mc'Donald
      • This is in accordance with RFC1778 which also disallows single quotes in all Relative Distinguished Name (RDN) components, and the OGF Certificate Authority Working Group (CAOPS) who strongly discourage any type of quote in a certificate DN as specified by their Grid Certificate Profile document.
  • Enter GOCDB5 central web portal at https://goc.egi.eu/portal

You can access the system as soon as you have a recognised X509 certificate, however you will only be able to update information if you register and obtain a role. More information about roles and associated permission is available in the #Users and roles section.

All roles applications need to be validated by parent roles or administrators. Once this is done, you can access/modify relevant information according to the role you have been granted. You can learn more on roles and user accounts by reading the #Users and roles section of this documentation.

How is the information organised?

GOCDB5 supports multiple projects. Each Project groups zero or more NGIs. An NGI groups zero or more Sites. A Site groups zero or more Services. ServiceGroups can also be used to group Services belonging to different Sites. Downtimes are declared over Services. Users have roles over target objects.

  • Projects group child NGI's
  • NGIs group chid Sites
  • Sites group child Services
  • Services group child Downtimes
  • Downtimes and related information
  • Service Groups Group Servies defined across different Sites
  • Users own zero or more Roles
  • Roles link a User to a target entity and a defined role type.


For more details see: https://wiki.egi.eu/w/images/d/d3/GOCDB5_Grid_Topology_Information_System.pdf

Users and roles

Understanding and manipulating user accounts

Authentication

The GOCDB UI attempts to authenticate you in one of two ways (the REST style API applies x509 only):

  • First, by requesting an IGTF accredited user certificate from your browser. If a suitable certificate is detected, you will be asked to confirm selection of your certificate in your browser.
  • Second, if you do not have a user certificate or you hide your certificate from GOCDB (e.g. by starting a new/anonymous private browser session or pressing 'Cancel' when prompted for a certificate), you will be redirected to the EGI Identity Provider Service (IdP) where you can authenticate with your chosen institution (if available). If authentication is successful, you will be re-directed back to GOCDB. Please note, not all logins available in the EGI IdP provide a sufficient level of assurance (LoA) to login to GOCDB (the LoA must be 'Substantial').

Each GOCDB user account is linked to a single account by an ID string - this ID from comes either your Certificate DN or from the EGI IdP service. It is important to note that GOCDB does not perform account-linking - each ID string maps to a separate GOCDB account. Existing users who have already registered an account will be logged into their account, while new users may choose to register a new account.

Registering a new user account

Being authenticated in one of the two ways described above is enough to have read-only access to all the public features of GOCDB. If you need to edit data in GOCDB and request roles, you will need to fill in the registration form.

To Register:

  • Go to the GOCDB web portal
  • In the left sidebar, look out for the User status panel
  • click on the "Register" link
  • fill in the form and validate

Note: If you were registered in GOCDB but are not recognised anymore (e.g. because your certificate DN changed), do not register again! Instead, follow the steps described in the #Changing_your_accountID section

Editing your user account

The editing process is the same as the registration process. To edit your use account, simply follow these steps:

  • click on the "view details" link in the "User Status" panel on the sidebar. you should get a page showing your user account information
  • click on the "edit" link on top of it.

Viewing users

Each user account has its own user details page which is accessible to anyone with a valid certificate.

There is currently no facility for listing all users in the database. List of users that have a role on a given site appears on site details pages (see section about sites). It is also possible to search for a user's account using the search feature on the sidebar.

Deleting your user account

If you wish to unregister from GOCDB, follow these steps:

  • click on the "view details" link in the "User Status" panel on the sidebar. you should get a page showing your user account information
  • click on the "delete" link on top of it.
  • confirm your choice

Your account will then be deleted along with any roles the account has.

Changing your accountID

Under the following circumstances it is possible to lose access to a GOCDB account that was originally created using a client certificate:

  • If you change your certificate, it is possible that the certificate's distinguished name (DN) has also changed. This is what GOCDB uses to identify your account.
  • If you choose to stop using your client certificate to log into GOCDB and istead access GOCDB via the EGI IdP.
  • If you have an account linked to your certificate but later login via the EGI-IdP route and mistakenly change your accountID from your certDN to the newly assigned ID issued by the EGI IdP.

In these situations, it is usually possible to regain access using to your certificate based account by following one of the following procedures:

If you have a new certificate and have lost access to your account

  • First install your new certificate in your browser.
  • Go to GOCDB. If you are already logged in, then clear your caches and restart your browser or start a new private browser session.
  • When prompted, select your new certificate but DON'T Register a new account.
  • You should be able to access GOCDB, but since you are authenticated with your new certificate, it is as if you had no user account (you have not registered your new certificate with GOCDB yet).
  • In the "user status" panel in the sidebar, click on the retrieve an old account link.
  • Specify in the form the DN of your old certificate, and the e-mail address associated to your account.
  • Upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account. This is to avoid identity theft. The e-mail contains a validation link.
  • Click on the validation link or copy/paste in your browser. Once validated, changes are immediate.

If you choose to stop using a client certificate in favour of the EGI IdP

NOTE: Following this process will mean you can *only* login to your GOCDB account via EGI Check-In going forward

  • Access GOCDB via the EGI IdP
  • In the "user status" panel in the sidebar, click on the retrieve an old account link.
  • Specify in the form: the DN of your old certificate; and the e-mail address associated to your account.
  • Upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account.
  • Click on the validation link or copy/paste in your browser. Once validated, changes are immediate.

If you mistakenly changed your accountID from your certDN to the ID issued from the EGI IdP and have lost access using your certificate

  • Go to GOCDB. If you are already logged in, then clear your caches and restart your browser or start a new private browser session.
  • When prompted, select your certificate you want to reinstate/re-associate with your account - DON'T Register a new account.
  • You should be able to access GOCDB but since you are authenticated with the certificate that is no longer linked to your account, it is as if you had no user account.
  • In the "user status" panel in the sidebar, click on the retrieve an old account link.
  • In the form, specify the DN of your certificate that you want to reinstate, and the e-mail address associated to your account.
  • Upon validation, an e-mail will be sent to the specified address, which has to match the one registered with your account. This is to avoid identity theft. The e-mail contains a validation link.
  • Click on the validation link or copy/paste in your browser. Once validated, changes are immediate.


If for any reason you were unable to complete these steps (e.g. mail confirmations problems) please do not register a new user account, but contact the GOCDB support helpdesk instead.

Understanding and manipulating roles

Roles definition

Registered users with a user account will need at least one role in order to perform any useful tasks.

Role Types

  • A role: Unregistered users
  • B role: Registered users with no role
  • C role: Users with a role at site level (site admin)
  • C' role: Users with a management role at site level (site operations manager, site security officer...)
  • D role: Users with a role at regional level (regional staff support staff, ROD, 1st Line Support)
  • D' role: Users with a management role at regional level (NGI manager or deputy, security officer)
  • E role: Users with a role at project level

The only difference between C and C' users is that:

  • C can NOT approve/reject role requests.
  • C' can only approve/reject role requests for their SITE.

The difference between D and D' users is that:

  • D can NOT add/delete sites to/from their NGI.
  • D can NOT update the certification status of member sites.
  • D can NOT approve or reject role requests.

Roles

  • At Site level
    • Site Administrator - person responsible of maintaining a grid site and associated information in GOCDB (C Level)
    • Site Security officer - official security contact point at site level (C' Level)
    • Site Operations Deputy Manager - The deputy manager of operations at a site (C' Level)
    • Site Operations Manager - The manager of site operations (C' Level)
  • At NGI/Regional level
    • Regional First Line Support - Staff providing first line support for an NGI (D Level)
    • Regional Staff (ROD) - staff involved in Operations Centre activities such as user/operations support (D Level)
    • NGI Security officer - official security contact point at regional level (D' Level)
    • NGI Operations Deputy Manager - Deputy manager of NGI operations (D' Level)
    • NGI Operations Manager - Manager of NGI operations (D' Level)
  • At Project level
    • COD staff - COD staff (E Level)
    • COD administrator - People administrating Central COD roles (E Level)
    • EGI CSIRT Officer - official security contact point at project level (E Level)
    • Chief Operations Officer (COO) - The EGI Chief Operations Officer (E Level)

Permissions associated to roles

GOCDB roles and permissions are based on whether the considered object is owned or not. In the table below the following definitions apply:

  • Owned group: a group on which the role applies (ROC, NGI, project)
  • Owned site: a site on which the role applies, or belonging to an owned group
  • Owned service endpoint: a service endpoint belonging to an owned site

Each role has a set of associated permissions which apply on the role's scope (site, region or project). Main permissions are summarised in the table below

Action A) Unregistered users B) Registered users with no role C) Site level users C' ) Site Management Level Users
D) NGI level users D' ) NGI Management Level Users E) Project level users
Add a site to an owned group irr. irr. irr. irr. no yes irr.
Add a site to a non owned group no no no no no no no
Add a service endpoint to an owned site irr. irr. yes yes yes yes irr.
Add a service endpoint to a non owned site no no no no no no no
Add a downtime to an owned service endpoint irr. irr. yes yes yes yes irr.
Add downtime to a non owned service endpoint no no no no no no no
Update information of an owned site irr. irr. yes yes yes yes irr.
Update information of a non owned site no no no no no no no
Update certification status of an owned site irr. irr. no no no yes yes
Update certification status of a non owned site no no no no no no yes
Update information of a owned service endpoint irr. irr. yes yes yes yes irr.
Update information of a non owned service endpoint no no no no no no no
Update information of an owned group irr. irr. irr. irr. yes yes irr.
Update information of a non owned group no no no no no no no
Update own user account details irr. yes yes yes yes yes yes
Update other user's account no no no no no no no
Update a downtime on an owned service endpoint irr. irr. yes yes yes yes irr.
Update a downtime on a non owned service endpoint no no no no no no no
Delete an owned site irr. irr. no no no no no
Delete a non owned site no no no no no no no
Delete an owned service endpoint irr. irr. yes yes yes yes irr.
Delete a non owned service endpoint no no no no no no no
Delete an owned group irr. irr. irr. no no no irr.
Delete a non owned group no no no no no no no
Delete a downtime on an owned service endpoint irr. irr. yes yes yes yes irr.
Delete a downtime on a non owned service endpoint no no no no no no no
Delete your own user account irr. yes yes yes yes yes yes
Delete other user's account no no no no no no no
Register a new user account yes irr. irr. irr. irr. irr. irr.
Request a new role no yes yes yes yes yes yes
Approve a role request on an owned group irr. irr. no no no yes yes
Approve a role request on an owned site no no no yes no yes irr
Approve a role request on a non owned site or group no no no no no no no
Reject a role request on an owned group no no no no no yes irr.
Reject a role request on an owned site no no no yes no yes irr
Reject a role request on a non owned site or group no no no no no no no
Revoke an existing role on an owned object irr. irr. no yes no yes irr.
Revoke an existing role on a non owned object no no no no no no no
Retrieve an existing account/ change certificate DN yes yes yes yes yes yes yes


Requesting roles for your account

There are 2 ways to request new roles.

  • By clicking on the manage role link (sidebar, user status panel)
    • the first form allows you to choose the entity (site or group) on which you want to request a role
    • the second form lets you choose the role you want to apply for
  • By clicking on the request role link from site detail pages or group detail pages.
    • displayed form lets you choose the role you want to apply for

Once made, role requests have to be validated before the role is granted to you. This part of the process is described in the next section.

Approving/revoking accounts, roles and other actions

Changing your certificate DN

Moved to: #Changing_your_accountID

Approving role and change requests

When a registered user applies for a role, the request has to be validated by someone who has the proper permissions to grant such a role. If you request a role on a given entity, any user with a valid role on that entity or above will be able to approve your request.

Example - If you request a "site administrator" role on site X, then the following users can approve your request:

  • site administrators and security officers of site X
  • regional operations staff, managers and deputies of the Operations Centre to which site X belongs
  • GOCDB admins

Role requests you can approve are listed on the Manage roles page (accessible by clicking the Manage roles link in the user status panel in the sidebar).

In order to approve or decline role requests, simply click on the accept or deny links in front of each role request.

Revoking roles

If a user within your scope has a role that needs to be revoked, you can do this from the user's page, where user's details are listed along with his/her current roles. To revoke a role, simply click on the role name then on the revoke link at the top right of the role's details page.

Note: This works for other users within your scope but also for yourself. However just note that if you revoke your own roles you may not have proper permissions to recover them afterwards.

NGIs (Site Group)

An NGI forms a grouping of Sites in GOCDB. GOCDB stores the following information about these groups. The main page listing groups actually shows NGIs/ROCs, and is available from

  • List of NGIs/ROCs and associated contacts, linked from the main menu


Each NGI has its own listing page, accessible by clicking on the "view" link in group listing pages. A group details page shows users with a role on that group, as well as member sites and associated contacts and roles.

Adding NGIs

Adding groups is not possible through the Input System web interface. If you want to start the registration process of a new NGI, please follow the procedure described on:

Integration of the new group in GOCDB is part of the procedure but has to be done by GOCDB admins.

Editing Groups

To edit a group, simply click on the "edit" link at the top of the group's details page.


Deleting Groups

This operation is not allowed.


Sites

Definition

A site (also known as a Resource Centre) is a grouping of grid resources collating multiple Service Endpoints (SEs). Down times are recorded on selected SEs of a site. GOCDB stores the following information about sites (non exhaustive list). Note, when editing values in the portal, mandatory fields are marked with '*':

  • A unique (short) name - case sensitive (GOCDB and GoCDB are considered different)
  • An official (long) name
  • A domain name for the Site/Resource Centre
  • The home web URL of the Site/Resource Centre
  • A contact email address and telephone number
    • Emergency e-mail for a fast response time in case of urgent problem
    • Alarm e-mail is WLCG Tier1 site specific (used as part of a WLCG workflow for dealing with specific monitoring alarms)
  • A security contact email address and telephone number
  • The site timezone
  • The site's GIIS URL (Case Sensitive - Please ensure you enter your Site name which is usually encoded in the URL in the correct case!).
    • e.g. ldap://bdii-rc.some-site.uk:2170/mds-vo-name=SITE-NAME,o=grid (if your GOCDB site name site name is upper case)
  • A mandatory human readable description of the site
  • The site's latitude, longitude and location
  • Production Infrastructure: The site's intended target infrastructure. This specifies the infrastructure that the site's services deliver to. This has one of the following values:
    • Production (with this target infrastructure, the EGI site certification transition rules apply)
    • Test (in future, if the site delivers to this infrastructure, then its Certification status will be fixed to 'Candidate').
  • ROC [GROUP] - The NGI or Region of the site
  • Country
  • IP address range within which the Site/Resource Centre's services run
    • IP/netmask (x.x.x.x/x.x.x.x). To specify multiple IP/netmask values, use a comma or semi-colon separated list with no spaces, e.g. 1.2.3.4/255.255.255.0,1.2.3.5/255.255.255.0

Manipulating sites

Viewing sites

A site listing page shows a listing of all the sites in the database, with controls to page through the listing. The table headers can be clicked to set the ordering (ascending or descending).

Each site also has its own listing page. By clicking the link to view a site, you can see all of the site's information

  • Site listing page is available from the sidebar by clicking on the Browse Sites link.
  • sites belonging to a given Operations Centre are also listed from the group details pages (see below)

Adding a site

Provided you have proper permissions (check the permissions matrix in the #Permissions_associated_to_roles section), you can add a site by clicking on the Add a New Site link in the sidebar. Simply fill the form and validate.

Note: If you just registered as site admin and want your new site to be registered in GOCDB, please contact your NGI representative.

Editing site information

The editing process will show you the same form as the adding process. To edit a site, simply click the "edit" link on top of the site's details page.

Renaming a site

Provided you have permissios, you can change the Short Name, Official Name and GIIS URL to the new Resource Center details. For more information regarding the site renaming procedure please see: PROC15

Removing a site

Site deletion is not allowed in GOCDB. If a site stops operation, its certification status should be set to "closed". See the section on #Changing_Site_Certification_Status for more information

Changing Site Certification Status

For each site that delivers to the 'Production' Target Infrastructure, GOCDB stores and shows information about its certification status. This reflects the different steps of the official SA1 site certification procedure which typically follows:

  • Candidate -> Uncertified -> Certified.

The different possible certification statuses are:

  • Candidate: the Resource Centre is in under registration according to the registration process described in the RC registration certification procedure. A site will have CANDIDATE status only during certification.
  • Uncertified: site information has been validated by the Operations Centre and is ready to be moved to certified status (again). The certification status of a site can only be changed by a user with a higher level 'Regional' (or EGI 'Project') level role. This usually means that only regional managers/deputies/staff can update the status of a site that belongs to that region, see #Permissions_associated_to_roles.
  • Certified: the Operations Centre has verified that the site has all middleware installed, passes the tests and appears stable.
  • Suspended: Site does temporarily not conform to production requirements (e.g. minimum service targets - see the Resource Centre OLA, security matters) and requires Operations Centre attention. A site can be suspended for a maximum of 4 months after which it must be re-certified or closed.
  • Closed: Site is definitely no longer operated by EGI and is only shown for historic reasons.


Clarifications:

  • The uncertified status would generally be an information that a site is ready to start certification procedure (again). "uncertified" can also be used as a timewise unlimited state for sites having to keep an old version of the middleware for the absolute needs of an important international VO or to flag a site coping with Operations Centre requirements but not with EGI availability/reliability thresholds.
  • Suspended is always having a temporary meaning. It is used to flag a site temporarily not coping with with EGI availability/reliability thresholds or security requirements, and which should be closed or uncertified by its Operations Centre within 4 months. When being suspended, sites can express that they want to pass certification again. The suspened status is useful to EGI and to the Operations Centre themselves to flag the sites that require attention by the Operations Centre.
  • The closed status should be the terminal one. Suspended is not a terminal state.


The following site state transitions are allowed:

  • candidate -> uncertified
  • candidate -> closed
  • uncertified -> certified
  • certified -> suspended
  • certified -> closed (on site request)
  • suspended -> uncertified
  • suspended -> closed


The following transitions are explicitly forbidden:

  • suspended -> certified
  • candidate -> something else but uncertified and closed
  • closed -> anything else


Going with the definition of the suspended status, Operations Centre managers have to regularly give their attention to all their suspended sites, so that they are processed within the given maximum time of four months. Sites being in suspended should either be set to closed or brought back in production via the uncertified status.

More information about site certification statuses can be found in SA1 certification and operation procedures documents:

Note: Site certification status cannot be changed by site administrators, and requires intervention of Operations Centre staff.



Defining Pay4Use Properties

Service Endpoints

Definition

A service endpoint is a single entity formed by a hostname, a hosted service and a URL.

GOCDB stores the following information about service endpoints (non exhaustive list):

* The fully qualified hostname of the machine
* The hosted service (see service types below)
* The URL to reach the endpoint
* The IP address of the machine
* The machine's host certificate DN
* A description of the node

As a machine can host many services, there can be many service endpoints per machine.

Example: the machine myhost.domain.org runs a CE, an UI and a UnicoreX service. This will show up in GOCDB as 3 Service Endpoints:

Note that a single host can also specify multiple services of the same service type.

Manipulating service endpoints

Viewing service endpoints

There are different pages in GOCDB where service endpoints are listed:

  • A full service endpoints listing page, that shows a listing of all the endpoints in the database, with controls to page through the listing. The table headers can be clicked to set the ordering.
  • Site details page, where all the service endpoints belonging to this site are listed

Each endpoint also has its own listing page. By clicking the link to view a service endpoint, you can see all associated information.

  • Service Endpoints listing page is available from the side menu in GOCDB4 by clicking on the Browse Service Endpoints link.

Adding Service Endpoints

There are 2 ways to add new service endpoints to GOCDB, provided you have proper permissions (check the permissions matrix in the #Permissions_associated_to_roles section):

  • By clicking on the Add a New Service link in the sidebar. Simply select parent site, fill the form and validate.
  • By clicking on the Add a New Service Endpoint link from a given site's details page (the link will only appear if you have proper permissions). This will lead you to the same form as above.

Editing service endpoint information

The editing process will show you the same form as the adding process. To edit a service endpoint, simply click the "edit" link on top of the endpoint's details page.

Removing a service endpoint from a site

to deactivate a service endpoint you have permissions on, simply clic on the "delete" link on top of the endpoint's details page. The interface asks for confirmation before proceeding.

Specific Service Endpoint fields and their impact

"beta" flag (t/f)

This indicates whether the service is a beta service or not (part of the staged rollout process). Beta is the equivalent at service level of the former EGEE Pre-Production Service (PPS)

Host DN

This is the DN of the host certificate for the service. The format of the DN follows that defined by the [OGF Interoperable Certificate Profile] which restricts allowed chars to a PrintableString that does NOT contain characters that cannot be expressed in printable 7-bit ASCII. For a list of allowed chars, see GFD.225.

To supply multiple or alternate DN(s) for a service, for example of the multiple hosts supporting a single service entry, see https://wiki.egi.eu/wiki/GOCDB/Input_System_User_Documentation#HostDN

"production" flag (t/f)

The services Production flag indicates if this service delivers a production quality service to the infrastructure it belongs to (EGI).

  • Non-production services can be either Monitored or Not Monitored, depending on the Administrator's choice.
  • Even if this flag is false, the service is still considered part of the EGI and so shows up in the dashboard.
  • If true, then the Monitored flag must also be true: All production resources MUST be monitored (except if the service type is a VOMS or emi.ARGUS)
  • This flag is not to be confused with PRODUCTION_STATUS, which is a Site level flag that shows if the site delivers to the production or Test infrastructure.

"monitoring" flag (t/f)

This flag is taken into account by monitoring tools.

  • Can only be set to "N" (false) if Production flag is also false.
  • If set to "N" the endpoint won't be tested.

Usage of PRODUCTION and MONITORED flags for EGI Service Endpoints

From 02/12/2014 all production services MUST be monitored (except for emi.ARGUS and VOMS service types).

Production and Monitored

  • Operations Dashboard: A failing test of production service endpoints generates an alarm in the ROD Operations Dashboard.
  • Availability calculation: The service endpoint test results are considered for Availability computation (if and only if the service type associated to the endpoint is one of those included in Availability computation)

Non-Production and Monitored: YES/NO

  • Availability calculation: If Monitored is set to YES, Service Availability Monitoring (SAM) will test the service, but SAM test results are ignored by the Availability Computation Engine (ACE).
  • Availability calculation: Non-production service endpoints are not considered for site availability calculations.
  • Operations Dashboard: If Monitored is set NO, the service endpoint is ignored by SAM and no alarms are raised in the Operations Dashboard in case of CRITICAL failure.
  • SAM tests for non-production services generate alarms into the ROD Operations Dashboard in case of CRITICAL failure of the test. These alarms are visible in the Operations Dashboard and are tagged as "non production".

Service Groups

A service group is an arbitrary grouping of existing service endpoints that can be distributed across different physical sites and users that belong to the SG (SGs were previously known as 'Virtual Sites'):

  • Each service that appears in a group must already exist and be hosted by a physical site.
  • A service group role does not extend any permissions over its child services. This means that you cannot declare a downtime on the services that you group together or modify the service attributes.
  • Any GOCDB user can create their own service group and as the 'Service Group Administator' you can control subsequent user membership requests to the SG (everything is logged, including who created the service group).
  • GOCDB users can request to join an existing service group by finding the target SG and requesting a role on that SG.
  • Service groups are typically used for monitoring a particular collection of services and/or users using the GOCDB 'get_service_group' and 'get_service_group_role' PI methods.
  • SG memebers can be listed using the get_service_group_role PI method.
  • PI doc:
  • If you have any further use-cases or suggestions, please submit a ticket to RT.


NGI Core Services

NGIs can register a number of ‘NGI-Core’ services in GOCDB. A core NGI service is one that is used to calculate the availability and reliability of the NGI. These services fall under the responsibility of the NGI and provide production quality (no testing instances). NGIs can distinguish/flag their core services from their other (non-core) services using one of two ways (see A and B below).

Core Service Requirements

The service instance MUST:

Required Service Types

The following service types are mandatory and all NGIs in the EGI scope should define instances of these services:

  • ngi.SAM (Mandatory)
  • emi.ARGUS (Mandatory) (NGI ARGUS)
  • Top-BDII (Mandatory)

Other Mandatory services, depending on middleware deployed by sites under NGI responsibility, are listed here

NGIs should also register their custom core services like accounting, helpdesk if they are registered in GOCDB (for a list of other common core service types see: https://wiki.egi.eu/wiki/NGI_services_in_GOCDB)

Registering NGI Core Services

NGI core services can be grouped/flagged in one of two ways:

  • A) By creating a ‘NGI_XX_SERVICES’ Site and adding their core services under this site. This site must be scoped as ‘NGI’ and define a certification status of ‘Certified’.
  • B) By creating a ‘NGI_XX_SERVICES’ ServiceGroup and adding their core services to this ServiceGroup.

It is important that these core service Sites/ServiceGroups adhere to the ‘NGI_XX_SERVICES’ naming scheme. For further details, including a list of existing ‘NGI_XX_SERVICES’ please see: https://wiki.egi.eu/wiki/NGI_services_in_GOCDB

Downtimes

Definition

A downtime is a period of time for which a service is declared to be inoperable. Downtimes may be scheduled (e.g. for software/hardware upgrades), or unscheduled (e.g. power outages). GOCDB stores the following information about downtimes (non exhaustive list):

  • The downtime classification (Scheduled or unscheduled)
  • The severity of the downtime
  • The date at which the downtime was added to GOCDB
  • The start and end of the downtime period
  • A description of the downtime
  • The entities affected by the downtime

Manipulating downtimes

Viewing downtimes

There are different pages on which downtimes are listed:

  • A "Recent and Planned" page , linked from the main menu, this provides a window to the EGI hosted information about downtimes. (Nb: This service requires users to add an exception to allow this window to render correctly)
  • A "Active & Imminent" page, linked from the main menu, that allows users to see currently active downtimes and downtimes planned in the coming weeks.
  • Site details page, where all the downtimes associated to the site are listed
  • Service endpoint details page, where all the downtimes associated to the service endpoint are listed
  • Service group details page, where all the downtimes associated to the service group are listed

Each downtime has its own listing page, accessible by clicking on the "view" link in downtime listing pages.

Adding downtimes

Provided you have proper permissions (check the permissions matrix in the #Permissions_associated_to_roles section), you can add a downtime by clicking on the Add a Downtime link in the sidebar.

This is done in 2 steps:

  • enter downtime information
  • specify the full list of impacted services in case there is more than one or select an site to select all the sites associated services.


Please note:

  • All dates have to be entered in UTC.
  • A downtime can be retrospectively added if its start-date is less than 48h in the past (giving a 2 day window to add).
  • downtime classification (scheduled/unscheduled) is determined automatically (see #Scheduled or unscheduled? section)

Editing downtime information

  • To edit a downtime, simply click the "edit" link on top of the downtime's details page.
  • A downtime can be retrospectively updated if its start-date is less than 48h in the past (giving a 2 day window to modify).
  • Note there are limitations to downtime editing, especially if it has already started, or is due to start in the next 24hrs or is finished. See #Downtime shortening and extension section for more details.

Removing downtimes

To delete a downtime, simply click the delete link on top of the downtime's details page. For integrity reasons, it is only possible to remove downtimes that have not started.

"Good practices" and further understanding

Scheduled or unscheduled?

Depending on the planning of the intervention, downtimes can be:

  • Scheduled: planned and agreed in advance
  • Unscheduled: planned or unplanned, usually triggered by an unexpected failure or at a short term notice

EGI defines precise rules about what should be declared as scheduled or unscheduled, based on how long in advance the downtime is declared. These rules are described in MAN02#How_to_manage_an_intervention and are enforced as follows:

  • All downtimes declared less than 24h in advance will be automatically classified as UNSCHEDULED
  • All other downtimes will be classified as SCHEDULED

Notes:

  • A downtime can be retrospectively declared and/or updated if its start-date is less than 48h in the past (giving a 2 day window to add/modify).
  • Although 24h in advance is enough for the downtime to be classified as "scheduled", it is good practice to declare it at least 5 working days before it starts.


WARNING or OUTAGE?

When declaring a downtime, you will be presented the choice of a "severity", which can be either WARNING or OUTAGE. Please consider the following definitions:

  • WARNING means the resource is considered available, but the quality of service might be degraded. Such downtimes generate notifications, but are not taken into account by monitoring and availability calculation tools. In case of a service failure during the WARNING period an OUTAGE downtime has to be declared, cancelling the rest of the WARNING downtime. (The WARNING flag now replaces the former AT_RISK flag).
  • OUTAGE means the resource is considered as unavailable. Such downtimes will be considered as "IN MAINTENANCE" by monitoring and availability calculation tools.

Downtime shortening and extension

Limitation rules to downtime extensions are enforced in GOCDB as follows:

  • Scheduled downtimes due to start in 24 hours cannot be edited in any way, but can be deleted.
  • Other downtimes that have not yet started can be edit and deleted.
    • They can be shortened or moved, i.e. They can be edited such that:
      • Both start and end time are still in the future
      • The duration remains the same or is decreased
  • Ongoing downtimes can not be deleted.
  • A downtime cannot be edited once it has finished, nor can a new downtime be added more than 48 hours into the past.

If for any reason a downtime already declared needs to be extended, the procedure is to add another adjacent downtime, before or after.

Service types

In GOCDB a service type is a technology used to provide a service. Each service endpoint in GOCDB is associated with a service type. Service types are pieces of software while service endpoints are a particular instance of that software running in a certain context.

Service Type Naming Scheme

  • Service types include grid middleware and operational services.
  • This attribute corresponds to the Glue2 'Service.Type' attribute and is defined as the 'Type of service according to a namespace based classification (the namespace MAY be related to a middleware name, an organisation or other concepts).'
  • The naming scheme for new service types in GOCDB therefore generally follow a reverse DNS style syntax, usually naming the technology provider/project followed by technology type in lowercase, i.e. ‘<provider>.<type>’ (e.g. ‘org.openstack.swift’).
  • Please note, this syntax does not necessarily indicate ownership, the main objective is to avoid name clashes between services. For example, different projects may have similar services but these may be modified/customised just enough to merit a different prefix or service type name.
  • Glue2 defines a service type list at: [Glue2 Enums] [Glue2 service types].
  • The Glue2 and GOCDB recommendation is to use lowercase (legacy enum values do exist that use camelCase).

These service types are used at some grid sites within EGI but aren't EGI operational tools or a part of the core middleware distributions (EMI, gLite, ARC, UNICORE, Globus etc).

Service Type List

To request a new service type, please submit a request for a new service type (described below).

Operational Components (middleware agnostic)

  • Site-BDII: [Site service] This service collects and publishes site's data for the Information System. All grid sites MUST install one Site-BDII. For cloud sites eu.egi.cloud.information.bdii MUST be installed.
  • Top-BDII: [Central service] The "top-level BDII". These collect and publish the data from site-BDIIs. Only a few instances per region are required.
  • MyProxy: [Central service] MyProxy is part of the authentication and authorization system. Often installed by sites installing the WMS service.
  • egi.APELRepository: [Central service] The central APEL repository
  • egi.AccountingPortal: [Central service] The central accounting portal
  • egi.GGUS: [Central service] The central GGUS
  • egi.GOCDB: [Central service] The central GOCDB
  • egi.MSGBroker: [Central service] The central message broker
  • egi.Portal: [Central Service] for monitoring generic web portals who dont have a specific service type
  • MSG-Broker: [Central service] A broker for the backbone messaging system.
  • egi.MetricsPortal: [Central service] The central metrics portal
  • egi.OpsPortal: [Central service] The central operations portal
  • egi.GRIDVIEW: [Central service] The central gridview portal
  • egi.GSTAT: [Central service] The central GStat portal
  • egi.SAM: [Central service] The central SAM monitoring
  • ngi.SAM: [Regional Service] NGI-level SAM monitoring box
  • vo.SAM: [Regional Service] VO-level SAM monitoring box
  • site.SAM: [Regional Service] Site-level SAM monitoring box
  • ngi.OpsPortal: [Regional service] NGI-level regional operations portal instance
  • eu.egi.MPI: Defines a dummy Service Type to enable the running of MPI tests for services providing MPI capabilities. Sites must have one instance of this Service Type associated with a CREAM-CE service. For details see https://wiki.egi.eu/wiki/VT_MPI_within_EGI:Nagios
  • argo.poem: POEM is system for managing profiles of probes and metrics in ARGO system.
  • argo.mon: ARGO Monitoring Engine gathers monitoring metrics and publishes to messaging service.
  • argo.consumer: ARGO Consumer collects monitoring metrics from monitoring engines.
  • argo.computeengine: ARGO Compute Engine computes availability and reliability of services.
  • argo.api: ARGO API service for retrieving status and A/R results.
  • argo.webui: ARGO web user interface for metric A/R visualization and recalculation management.
  • egi.aai.saml: EGI AAI CheckIn SAML interface. Enables federated access to EGI services and resources using Security Assertion Markup Language (SAML). Provided by GRNET.
  • egi.aai.oidc: EGI AAI CheckIn OpenID Connect interface. Enables federated access to EGI services and resources using OpenID Connect (OIDC). Provided by GRNET.
  • egi.aai.tts: EGI AAI CheckIn token translation service. Enables the translation between different authentication and authorisation protocols. Provided by GRNET.


Middleware (ARC, gLite, Unicore)
ARC Middleware

  • ARC-CE: [Site service] The Compute Element within the ARC middleware stack.
  • SGAS: [Site service] An accounting service used by ARC.
  • org.nordugrid.arex: [Site Service] ARC version 3 Compute element.

gLite Middleware

  • CE: [OBSOLETE Site service] The LCG Compute Element. Currently the standard CE within the gLite middleware stack. Replaced by the CREAM CE.
  • gLite-CE: [OBSOLETE Site service] The gLite Compute Element is now obsolete and is not supported. Please avoid using this middleware service.
  • CREAM-CE: [Site service] The CREAM Compute Element is the new CE within the gLite middleware stack.
  • APEL: [Site service] This is a "dummy" Service Type to enable the monitoring tests for APEL accounting. All sites must have one instance of this Service Type, associated with a CE.
  • MON: [OBSOLETE Site service] The gLite MonBox hosts the site R-GMA services.
  • UI: [User service] The User Interface. Can be installed by users but more commonly installed by a site.
  • SRM: [Site service] Storage Resource Manager. Mandatory for all sites running an SRM enabled storage element.
  • SRM.nearline: [Site service] Storage Resource Manager for tape only.

* SRM.online: [Site service] Storage Resource Manager for disk only.

  • Classic-SE: [OBSOLETE Site service] The Classic Storage Element is now obsolete and is not supported. Please avoid using this middleware service.
  • Central-LFC: [Central service] An instance of the gLite file catalogue which holds entries for all files owned by a particular VO. NOTE: An LFC can be both Central and Local.
  • Local-LFC: [Site service] An instance of the gLite file catalogue which holds entries for files owned by a particular VO, at your site. NOTE: An LFC can be both Central and Local.
  • WMS: [Central service] gLite Workload Management Service. Acts as the broker for matching user jobs to available computing resources.
  • RB: [OBSOLETE Central service] The LCG Resource Broker is now obsolete and is not supported. Please avoid using this middleware service.
  • VOMS: [Central service] VO Management System. Part of the authentication and authorization system. This service only needs to be installed on the request of a VO.
  • LB: [Central service] gLite Logging and Bookkeeping. Usually installed by sites running a WMS. One LB service can support several WMS instances.
  • AMGA: [Central service] gLite metadata catalogue. This service only needs to be installed on the request of a VO.
  • FTM: [Site service] gLite File Transfer Monitor. Monitors the FTS service at a site.
  • FTS: [Central service] The gLite File Transfer Service manages the transfer of files between sites. This service only needs to be installed on the request of a VO.
  • VO-box: [Site service] The gLite VO box allows a VO to run their own services at a site. This service only needs to be installed on the request of a VO.
  • gLite-APEL: [Site service] The gLite-APEL hosts the site Accounting client (3.2 replacement of the MonBox)
  • gLExec: [Site service] A light-weight gatekeeper to authenticate and authorize credentials according to local site policy and execute commands. https://www.nikhef.nl/pub/projects/grid/gridwiki/index.php/GLExec
  • emi.ARGUS: [Site service] The Argus Authorization Service renders XACML authorization decisions for distributed services, based on policies
  • ngi.ARGUS: [Central Service] Distinguishes site Argus instances (emi.ARGUS) from NGI ones (ngi.ARGUS). While servicegroups can be used for this in gocdb, this is needed for backward-compatibility with the monitoring.

Unicore Middleware

  • eu.unicore.UnicorePortal UNICORE Portal service allows users to access a UNICORE Grid via the web browser. Service be used by users instead of standalone clients like UCC and URC to access resources and submit jobs. The UNICORE Portal server is available for download at UNICORE site: http://unicore.eu/download/unicore6/. It is possible in the future that each NGI supporting UNICORE middleware will deploy at least one instance.
  • unicore6.Registry: [Central service] All UNICORE services register here; clients ask the registry for available services in the Grid. Normally one Registry per Grid infrastructure which collects URLs of services.
  • unicore6.Gateway: [Site service] Sits in front of one or more UNICORE services as a gateway to the internet. Normally one Gateway per site.
  • unicore6.TargetSystemFactory [Site service] used as an entry-point for submitting single jobs. It can create Target System Services (TSSs) and submit jobs to those TSSs.
  • unicore6.StorageFactory [Site service] Creates StorageManagement instances. A user can create dynamic storage management services for own purposes with it. Often used to provide filespace during workflow execution.
  • unicore6.StorageManagement [Site service] Provides an abstract filesystem-like view on a storage resource. A Storage Management Service (SMS) can be created by a Storage Factory or can be configured statically way by a config file.
  • unicore6.ServiceOrchestrator [Site service] Handles dispatching of a workflow's atomic jobs, and brokering. Normally there is one per grid infrastructure.
  • unicore6.WorkflowFactory [Site service] Used as an entrypoint for submitting workflow jobs. The Workflow factory is creating workflow instances and can submit workflows to them. It is the workflow submission equivalent to the Target System Factory used for single job submission.
  • unicore6.UVOSAssertionQueryService [Site service] Provides data and user information via the SAML standard as needed for authorization and environment customization.

Globus Middleware

  • GRAM5: [Site service] job submission service for Globus version 5.x (GRAM5).
  • globus-GRIDFTP: [Site service] storage endpoint and data transfer service for the Globus middleware stack.
  • globus-GSISSHD: [Site service] certificate based interactive login service for the Globus middleware stack.
  • MyProxy: [Site service] MyProxy is part of the authentication and authorization system.

QosCosGrid (QCG) Middleware

  • QCG.Computing [Site service] A compute component based on the OGF Basic Execution Service (BES) with advanced reservation support.
  • QCG.Notification [Site service] A notification middleware component using a brokered version of the OASIS WS-Notification standard.
  • QCG.Broker [Site service] QosCosGrid resource management and brokering service.

EDGI Middleware (European Desktop Grid Initiative)

  • dg.CREAM-CE CREAM gateway to Desktop Grid
  • dg.3GBridge The 3G Bridge (Generic Grid Grid Bridge) is designed to be used as a mediator between different types of Grid middleware.

Cloud

Other

* ch.cern.cvmfs.stratum.1 Service component (stratum.1) of CernVM file system http://cernvm.cern.ch/portal/filesystem

  • uk.ac.gridpp.vac A virtual machine factory system for operating clusters at grid sites
  • hu.guse.ws-pgrade WS-PGRADE/gUSE-based science gateway deployment
  • org.opensciencegrid.htcondorce A special configuration of the HTCondor software designed to be a job gateway solution
  • uk.ac.gridpp.vcycle Vcycle manages the lifecycle of VMs running jobs on cloud resources. LHCb, ATLAS, CMS, and GridPP DIRAC VMs
  • webdav Service with extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers.

Custom Service Types
In order to control the proliferation of custom service types, please consider submitting a request for a new service type (described below) before using CUSTOM_SERVICE.

* CUSTOM.pl.plgrid.Bazaar SLA negotiation system between users and resource providers from NGI_PL grid

Adding new services types

Please feel free to make a request for a new service type. For CUSTOM service types, we would like to make this process as light-weight as possible. However, currently all new service type requests need to be assessed by EGI via lightweight review process (by OMB and EGI Ops) so that only suitable types are added to GOCDB and to prevent duplication.

You can submit your request via GGUS to the "Configuration and Topology Database (GOCDB)" support unit.

Please specify the following information as part of your request:

- name of service type (lowercase):
- high-level description of the service functionality (255 characters max):
- project/community/organization maintaining the software:
- scale of deployment (number of instances and by which organizations):
- contact point (name/e-mail address):

Note - please provide a suggested SE type name following the naming scheme described above (technology provider's reversed domain . software name) and a brief sentence to describe the service type.

Guidelines here for adding custom service types to SAM for monitoring.

Data Visibility / Scopes

  • Scope tags are used to group Grid entities such as Sites, Services and ServiceGroups into flexible categories. A single entity can define multiple scope tags, allowing the resource to be associated with different categories without duplication of information. This is essential to maintain the integrity of topology information across different infrastructures and projects.
  • The GOCDB admins control which scope tags are made available to avoid proliferation of tags (user defined tags are reserved for the extensibility mechanism).
  • As an example, a site’s scope list could aggregate all of the scopes defined by its child services. In doing this, the site scope list becomes a union of its service scopes plus any other site specific tags defined by the site.
  • By defining scope tags, resources can be ‘filtered-by-scope-tag’ when querying for data in the PI using the ‘scope’ and ‘scope_match’ parameters, see GOCDB Programmatic Interface (GOCDB-PI) for details.

Clear Separation of Concerns

It is important to understand that scopes and Projects are distinct:

  • Projects are used to cascade roles and permissions over child objects
  • Scope tags are used to filter resources into flexible categories/groupings
  • Scope tags can be created to mirror the projects. For example, assuming two projects (e.g. EGI.eu and EUDAT), two corresponding tags may be defined.
  • In addition, it is also possible define additional scopes for finer grained resource filtering e.g. ‘SubGroupX’ and ‘EGI_TEST’.
  • The key benefit: A clear separation of concerns between cascading permissions and resource filtering.

EGI Scopes

  • To make a Site, Service or ServiceGroup visible to EGI, the resource's 'EGI' scope tag check box must be ticked. EGI scoped resources are exposed to the central operational tools for monitoring and will appear in the central operations portal.
  • Un-ticking the EGI checkbox and selecting the 'Local' scope makes the selected object invisible to EGI; it will be hidden from the central operation tools (it will not show in the central dashboard and it will not be monitored centrally). This can be useful if you wish to hide certain parts of your infrastructure from EGI but still have the information stored and accessed from the same GOCDB instance.
  • A use-case for non-EGI sites/services is to hide those entities from central EGI tools, but to include those sites/services for use by regional versions of the operational tools (such as regional monitoring). To enable regional monitoring of non-EGI sites/services using SAM see [original change request] and [Add support for GOCDB scope]
  • Note that exposing a site / service endpoint as EGI does not override the production status or certification status fields. For example if a site isn't marked as production it won't be monitored centrally even if it's marked as visible to EGI.
  • You can submit your request for new scope tags via GGUS to the "Configuration and Topology Database (GOCDB)" support unit.

Reserved Scope Tags

  • Some tags may be 'Reserved' which means they are protected - they are used to restrict tag usage and prevent non-authorised sites/services from using tags not intended for them.
  • Reserved tags are initially assigned to resources by the gocdb-admins, and can then be optionally inherited by child resources (tags can be initially assigned to NGIs, Sites, Services and ServiceGroups).
  • When creating a new child resource (e.g. a child Site or child Service), the scopes that are assigned to the parent are automatically inherited and assigned to the child.
  • Reserved tags assigned to a resource are optional and can be de-selected if required.
  • Users can reapply Reserved tags to a resource ONLY if the tag can be inherited from the parent Scoped Entity (parents include NGIs/Sites).
    • For Sites: If a Reserved tag is removed from a Site, then the same tag is also removed from all the child Services - a Service can't have a reserved tag that is not supported by its parent Site.
    • For NGIs: If a Reserved tag is removed from an NGI, then the same tag is NOT removed from all the child Sites - this is intentionally different from the Site->Service relationship.
  • To request a reserved scope tag, an approval is required from the operators of the relevant resources. Details on who to contact are listed below. Once authorisation is given, please contact the GOCDB admins with details of the approval (e.g. link to a GGUS ticket that approves the tag assignment).

FedCloud Reserved Tag

  • Tag for resources that contribute to the EGI Federated Cloud. To request this tag, please contact the FedCloud operators / EGI Operations.

Elixir Reserved Tag

  • Tag for resources that contribute to the EGI Federated Cloud. To request this tag, please contact the operators of the ‘ELIXIR’ NGI in GOCDB.

WLCG Reserved Tags

  • A number of reserved scope tags have been defined for the WLCG:
    • The ‘tierN’ tags should be requested for WLCG sites that are defined in REBUS (a management view of the WLCG infrastructure/sites). To request a ‘tierN’ tag, raise a ticket against the REBUS support unit in GGUS.
    • For the experiment VO tags (alice, atlas, cms, lhcb), raise a ticket with the relevant VO support unit.
    • The wlcg tag is a generic catch-all tag for sites/services with either tierN and VO tags and is used to gain an overall view of the WLCG infrastructure.

SLA Reserved Tag

  • Entitities covered by an EGI VO SLA
    • This Tag will only be applied at the request of EGI operations

Extension Properties

NOTE: From GOCDB 5.7 (Autumn/Winter 2016) keys must be unique for a given site, service, or service endpoint, or service group.

  • Sites, Services, service endpoints, and Service Groups can be extended by adding custom key-value pairs (this follows the GLUE2 extensibility mechanism).
  • Extension properties address a number of use cases, such as filtering Sites and/or Services that define particular properties, e.g. 'charging rate': https://rt.egi.eu/rt/Ticket/Display.html?id=3764
  • Selected methods in the GOCDB PI support the 'extensions' URL parameter. This parameter is used to filter resources according to the extensions they define (described below).
  • Properties are rendered in the XML results of the Site/Service/ServiceGroup using the <EXTENSIONS> XML element, for an example see a sample output from get_service_endpoint
  • Note, anyone with permissions over the target entity can add extension properties to that object.
  • This allows 'Folksonomy' building: 'a user-generated system of classifying and organizing content into different categories by the use of metadata such as electronic tags'
  • A number of use cases can be addressed; e.g. filtering Sites that support a specific property, e.g. ‘P4U_Pilot_Cloud_Wall’
  • Key/value pairs (currently) prevent certain characters from being used in their values. This includes the equals and opening/closing parenthesis chars ‘=()’. This is to simplify lexical parsing of the query. In addition, to guard against cross-site scripting attacks, the quote, double quote, semi-colon and back tick chars are also not allowed.

Extension Properties in the PI

  • Selected PI methods allow results to be filtered by extension properties via the 'extensions' PI parameter.
  • Supported methods include: get_site, get_site_list, get_service_endpoints and get_service_group, get_downtime, get_downtime_nested, get_site_list.
  • For individual method support please refer to the PI documentation: GOCDB/PI/Technical Documentation
  • The format of the 'extensions' PI parameter is one or more (key=value) pairs enclosed in brackets.
    • The value part of a (k=v) pair can be ommitted if filtering by value is not required (i.e. '(somekey=)' means select all resources that define the 'somekey' property with any value.
    • (k=v) pairs can be optionally prefixed with one of following operators: AND, OR, NOT.
    • If no operator is specified before the FIRST (k=v) pair, then AND is assumed.
    • A single operator applies to ALL the (k=v) pairs to the right of the operator until another operator is encountered.
    • An AND forms a logical conjunction with any previously specified conditions.
    • An OR forms a logical disjunction with any previously specified conditions.
    • A NOT forms a logical conjunction with any previously specified conditions (it can be read as 'AND NOT')
    • Because an OR always forms a logical disjunction with any previously specified conditions, you can’t OR against a group occurring to the right that contains multiple k=v pairs e.g. the following is not supported (if there is sufficient demand, it could be considered for a future enhancement):
      • ((k=v1)AND(k=v2)) OR ((k=v3)AND(k=v4))


Examples:

  • Eg (note no leading AND):
    • (key1=val)(key2=va2)OR(key3=val3)(key4=val4)NOT(key5=val5)(key6=val6)   is expanded to:
    • AND(key1=val)AND(key2=va2)OR(key3=val3)OR(key4=val4)NOT(key5=val5)NOT(key6=val6)    which is interpreted as:
    • (((key1=val)AND(key2=va2))OR(key3=val3)) OR(key4=val4) NOT(key5=val5) NOT(key6=val6)
  • Eg:
    • (VObing=true)AND(VObaz=true)AND(VObar=true)OR(s1p1=v1)      is equal to:
    • is equal to: ((VObing=true)AND(VObaz=true)AND(VObar=true))OR(s1p1=v1)
  • Eg:
    • (VO=food)OR(VO2=bar)AND(s4p1=v1)       is equal to:
    • ((VO=food)OR(VO2=bar))AND(s4p1=v1)
  • Eg:
    • (VO=food)(s4p1=v1)OR(VObar=true)(VObaz=true)       is equal to:
    • is equal to: ((VO=food)AND(s4p1=v1))OR(VObar=true)OR(VObaz=true)
  • Eg:
    • (VO=food)(s4p1=v1)OR(VObaz=true)AND(VObling=true)       is equal to:
    • (((VO=food)AND(s4p1=v1))OR(VObaz=true))AND(VObling=true)


To return all sites that define VO with a value of Alice:

?method=get_site&extensions=(VO=Alice)

Use no value to define a wildcard search, i.e. all sites that define the VO property regardless of value:

?method=get_site&extensions=(VO=)

NOTE: From GOCDB 5.7 (Autumn/Winter 2016) keys must be unique for a given site, service, or service endpoint, or service group. The following section of documentation has not yet been changed to reflect this.

Extensions also supports OR/AND/NOT operators. This can be used to search against multiple key values eg:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)

These can be used together:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)
?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)OR(CPU_HS02_HOUR=2)

When no operator is specified the default is AND, therefore the following:

?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

Is the same as:

?method= get_service_endpoint&extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

The extensions parameter can also be used in conjunction with the existing parameters previously supported:

?method=get_site&extensions=(VO=Alice)NOT(VO=LHCB)&scope=EGI&roc=NGI_UK
  • The 'site_extensions' and 'service_extensions' can also be used on the 'get_downtime' and 'get_downtime_nested_services' methods using same logic described above. Note, the <EXTENSIONS> element is not rendered in the XML output for these queries.
?method=get_downtime_nested_services&site_extensions=(eg.2=val.2)&service_extensions=(eg.2=)
?method=get_downtime&site_extensions=(eg.2=val.2)&service_extensions=(eg.2=)

Standard Extension Properties

HostDN

From GOCDB 5.8.0 onwards

For EGI Services, the Standard Extension property "HostDN" has been defined to allow the fetching the DNs of multiple hosts behind a load balanced service from the endpoint properties of a single GOCDB Service, rather than creating multiple GOCDB Services with different host DNs.

Recommended Use

To supply multiple or alternate DN(s) for a service, for example of the multiple hosts supporting a single service entry, the Service Extension Property (hereafter Ext) "HostDN" SHOULD be used. If Ext "HostDN" is present it MUST contain one or more x.509 DN values. Multiple values MUST be delimited by enclosing each within <> characters. If Ext "HostDN" is present, the Service "Host DN" SHOULD contain the x.509 SubjectAltName used in the x.509 certificate(s) presented by the hosts identified by the Ext "HostDN" values.

Options for adding a new Project in GocDB

GocDB is multi-tenanted; it can host multiple projects in the same instance. There are a number of different deployment scenarios that can be used to support new projects detailed below. Please contact the GocDB admins/EGI Ops to request the addition of a new project.

1) Add resources (sites/services) to an existing project

  • Resources (NGIs, Sites, Services) would be hosted under an existing project, e.g. the ‘EGI’ project.
  • The new resources would be subject to the rules/roles of the existing project, such as site certification status changes and project controlled user memberships.
  • The resources could not be filtered using a custom scope tag.

2) Add resources (sites/services) to an existing project and add a new Scope tag to represent a sub-grouping

  • Resources would be hosted under an existing project, and a new scope tag would be added for the purposes of resource filtering.
  • Since the resources are still hosted under an existing project, the resources would still be subject to the rules/roles of that project such as project controller user memberships.
  • The resources could be filtered using the new scope tag, but this scope tag would not strictly represent a project, rather a sub-grouping under the existing project, e.g.
    get_services&scope=SubGroupX
    Note, resources can be tagged multiple times to declare support for multiple projects and sub-groups:
    get_services&scope=SubGroupX,EGI&scope_match=all

3) Add resources (sites/services) to a new Project and add a new Scope tag to filter by project

  • Resources would be hosted under a new project, and a new scope tag would be added named after the project for the purposes of resource filtering.
  • The resources would not be subject to the rules/roles of other projects, for example, allowing the project to control its own project memberships.
  • The resources could be filtered using the scope tag named after the new project, e.g.
    get_services&scope=ProjectX
    Note, resources can be tagged multiple times to declare support for multiple projects:
    get_services&scope=ProjectX,EGI&scope_match=all

How to and FAQ

I get an "error 12227" message when accessing GOC portal with Mozilla/Firefox

This happens when no certificate has been uploaded to your browser. Refer to the "Access to GOCDB" section for more information about GOCDB and X509 certificates.

I am responsible for a site that has recently entered the EGI infrastructure. How do I register it?

Only registered users with an approved role on an NGI can add a new site. If you are the site administrator, the first thing to do is to contact your NGI staff and ask them to add the site for you. Then, register to GOCDB (see the user account section) and ask for a site admin role for your site (see the requesting a role section). Once your role approved, you will be able to edit and change your site information.

Why can't I declare downtimes for my whole site as I used to do in GOCDB3?

For data clarity reasons, it has been decided long ago to only link downtimes to services, thus avoiding the complication of having to check both site and service downtimes to determine whether a service is up or not. The way to declare a downtime for your site is to select all the services of the site in one go when inserting the downtime.

How do I extend a declared schedule downtime?

Because of EGI policies it is not possible to extend a downtime. Recommended good practice for any downtime extension is to declare a new unscheduled downtime, starting just when the frst one finishes. please refere to the downtimes section of this documentation for more information, especially the "downtime extension" paragraph.

I have declared a downtime "at risk", and it turns out to be an outage. How can I declare this properly?

If you have declared the downtime as being at risk and an outage actually happens half way through, you need to update GOCDB to reflect the fact that your site is now down. There is currently no way of doing this by updating the downtime on the fly without having the system considering the whole downtime as being an outage. The best way to proceed is:

  • Modify end date of your "at risk" downtime, so that it ends in a few minutes
  • Enter a new "outage" downtime, starting when the other ends

How do I switch monitoring on/off for my nodes?

Monitoring status in GOCDB cannot always be switched off. If a node is declared as delivering a production service, rules apply and the node has to be monitored. If you are running a test node and want to switch monitoring off, set both "monitoring" and "production" to "N".

Why nobody has approved my role request yet?

Someone has to approve any request you make, in order to ensure nobody is trying to get inappropriate roles. If yours is not getting approved, this can either be because your request was not legitimate, or most likely because the people that are supposed to do it forgot about it. Please refer to the Roles permissions definitions section of this documentation to determine who should validate your role, and try to get in touch with them. If you are requesting a site admin role, they are likely to be your fellow site admins or your NGI operators.

I am not an EGI user but need access to GOCDB backend to retrieve information for my project. What can I do?

Accessing GOCDB backend through another way than the GOC portal web interface is out of the scope of this documentation. please refer to the technical documentation instead, which is available from  GOCDB Documentation Index.