GOCDB/Input System User Documentation

From EGIWiki
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


Contents

Introduction

Scope of this documentation

This user documentation is about the GOCDB5 Input System, which is either:

Other documentation

Version and improvements

This documentation is meant to be useful and accurate. If you think it is not, please send us any improvement suggestions to gocdb-admins_at_mailman.egi.eu

GOCDB version supported in this documentation: 5.3+

Quick Orientation guide

Accessing GOCDB5 input system

To access the web interface, you need an X509 digital certificate installed in your browser, delivered by one of the recognised EU-Grid-PMA Certification Authorities.

You can access the system as soon as you have a recognised X509 certificate, however you will only be able to update information if you register and obtain a role. More information about roles and associated permission is available in the #Users and roles section.

All roles applications need to be validated by parent roles or administrators. Once this is done, you can access/modify relevant information according to the role you have been granted. You can learn more on roles and user accounts by reading the #Users and roles section of this documentation.

How is the information organised?

GOCDB5 supports multiple projects. Each Project groups zero or more NGIs. An NGI groups zero or more Sites. A Site groups zero or more Services. ServiceGroups can also be used to group Services belonging to different Sites. Downtimes are declared over Services. Users have roles over target objects.


For more details see: https://wiki.egi.eu/w/images/d/d3/GOCDB5_Grid_Topology_Information_System.pdf

Users and roles

Understanding and manipulating user accounts

Authentication

The GOCDB UI attempts to authenticate you in one of two ways (the REST style API applies x509 only):

Each GOCDB user account is linked to a single account by an ID string - this ID from comes either your Certificate DN or from the EGI IdP service. It is important to note that GOCDB does not perform account-linking - each ID string maps to a separate GOCDB account. Existing users who have already registered an account will be logged into their account, while new users may choose to register a new account.

Registering a new user account

Being authenticated in one of the two ways described above is enough to have read-only access to all the public features of GOCDB. If you need to edit data in GOCDB and request roles, you will need to fill in the registration form.

To Register:

Note: If you were registered in GOCDB but are not recognised anymore (e.g. because your certificate DN changed), do not register again! Instead, follow the steps described in the #Changing_your_accountID section

Editing your user account

The editing process is the same as the registration process. To edit your use account, simply follow these steps:

Viewing users

Each user account has its own user details page which is accessible to anyone with a valid certificate.

There is currently no facility for listing all users in the database. List of users that have a role on a given site appears on site details pages (see section about sites). It is also possible to search for a user's account using the search feature on the sidebar.

Deactivating a user account

If you wish to unregister from GOCDB, follow these steps:

Your account will then be deactivated and all your roles revoked.

Changing your accountID

Under the following circumstances it is possible to lose access to a GOCDB account that was originally created using a client certificate:

In either of these situations, it is usually possible to revert and regain access using your certificate by following one of the following procedures:

If you have a new certificate and have lost access to your account

If you mistakenly changed your accountID from your certDN to the ID issued from the EGI IdP and have lost access using your certificate


If for any reason you were unable to complete these steps (e.g. mail confirmations problems) please do not register a new user account, but contact the GOCDB support helpdesk instead.

Understanding and manipulating roles

Roles definition

Registered users with a user account will need at least one role in order to perform any useful tasks.

Role Types

The only difference between C and C' users is that:

The difference between D and D' users is that:

Roles

Permissions associated to roles

GOCDB roles and permissions are based on whether the considered object is owned or not. In the table below the following definitions apply:

Each role has a set of associated permissions which apply on the role's scope (site, region or project). Main permissions are summarised in the table below

Action A) Unregistered users B) Registered users with no role C) Site level users C' ) Site Management Level Users
D) NGI level users D' ) NGI Management Level Users E) Project level users
Add a site to an owned group irr. irr. irr. irr. no yes irr.
Add a site to a non owned group no no no no no no no
Add a service endpoint to an owned site irr. irr. yes yes yes yes irr.
Add a service endpoint to a non owned site no no no no no no no
Add a downtime to an owned service endpoint irr. irr. yes yes yes yes irr.
Add downtime to a non owned service endpoint no no no no no no no
Update information of an owned site irr. irr. yes yes yes yes irr.
Update information of a non owned site no no no no no no no
Update certification status of an owned site irr. irr. no no no yes yes
Update certification status of a non owned site no no no no no no yes
Update information of a owned service endpoint irr. irr. yes yes yes yes irr.
Update information of a non owned service endpoint no no no no no no no
Update information of an owned group irr. irr. irr. irr. yes yes irr.
Update information of a non owned group no no no no no no no
Update own user account details irr. yes yes yes yes yes yes
Update other user's account no no no no no no no
Update a downtime on an owned service endpoint irr. irr. yes yes yes yes irr.
Update a downtime on a non owned service endpoint no no no no no no no
Delete an owned site irr. irr. no no no no no
Delete a non owned site no no no no no no no
Delete an owned service endpoint irr. irr. yes yes yes yes irr.
Delete a non owned service endpoint no no no no no no no
Delete an owned group irr. irr. irr. no no no irr.
Delete a non owned group no no no no no no no
Delete a downtime on an owned service endpoint irr. irr. yes yes yes yes irr.
Delete a downtime on a non owned service endpoint no no no no no no no
Delete your own user account irr. yes yes yes yes yes yes
Delete other user's account no no no no no no no
Register a new user account yes irr. irr. irr. irr. irr. irr.
Request a new role no yes yes yes yes yes yes
Approve a role request on an owned group irr. irr. no no no yes yes
Approve a role request on an owned site no no no yes no yes irr
Approve a role request on a non owned site or group no no no no no no no
Reject a role request on an owned group no no no no no yes irr.
Reject a role request on an owned site no no no yes no yes irr
Reject a role request on a non owned site or group no no no no no no no
Revoke an existing role on an owned object irr. irr. no yes no yes irr.
Revoke an existing role on a non owned object no no no no no no no
Retrieve an existing account/ change certificate DN yes yes yes yes yes yes yes


Requesting roles for your account

There are 2 ways to request new roles.

Once made, role requests have to be validated before the role is granted to you. This part of the process is described in the next section.

Approving/revoking accounts, roles and other actions

Changing your certificate DN

Moved to: #Changing_your_accountID

Approving role and change requests

When a registered user applies for a role, the request has to be validated by someone who has the proper permissions to grant such a role. If you request a role on a given entity, any user with a valid role on that entity or above will be able to approve your request.

Example - If you request a "site administrator" role on site X, then the following users can approve your request:

Role requests you can approve are listed on the Manage roles page (accessible by clicking the Manage roles link in the user status panel in the sidebar).

In order to approve or decline role requests, simply click on the accept or deny links in front of each role request.

Revoking roles

If a user within your scope has a role that needs to be revoked, you can do this from the user's page, where user's details are listed along with his/her current roles. To revoke a role, simply click on the role name then on the revoke link at the top right of the role's details page.

Note: This works for other users within your scope but also for yourself. However just note that if you revoke your own roles you may not have proper permissions to recover them afterwards.

NGIs (Site Group)

An NGI forms a grouping of Sites in GOCDB. GOCDB stores the following information about these groups. The main page listing groups actually shows NGIs/ROCs, and is available from


Each NGI has its own listing page, accessible by clicking on the "view" link in group listing pages. A group details page shows users with a role on that group, as well as member sites and associated contacts and roles.

Adding NGIs

Adding groups is not possible through the Input System web interface. If you want to start the registration process of a new NGI, please follow the procedure described on:

Integration of the new group in GOCDB is part of the procedure but has to be done by GOCDB admins.

Editing Groups

To edit a group, simply click on the "edit" link at the top of the group's details page.


Deleting Groups

This operation is not allowed.


Sites

Definition

A site (also known as a Resource Centre) is a grouping of grid resources collating multiple Service Endpoints (SEs). Down times are recorded on selected SEs of a site. GOCDB stores the following information about sites (non exhaustive list). Note, when editing values in the portal, mandatory fields are marked with '*':

Manipulating sites

Viewing sites

A site listing page shows a listing of all the sites in the database, with controls to page through the listing. The table headers can be clicked to set the ordering (ascending or descending).

Each site also has its own listing page. By clicking the link to view a site, you can see all of the site's information

Adding a site

Provided you have proper permissions (check the permissions matrix in the #Permissions_associated_to_roles section), you can add a site by clicking on the Add a New Site link in the sidebar. Simply fill the form and validate.

Note: If you just registered as site admin and want your new site to be registered in GOCDB, please contact your NGI representative.

Editing site information

The editing process will show you the same form as the adding process. To edit a site, simply click the "edit" link on top of the site's details page.

Renaming a site

Provided you have permissios, you can change the Short Name, Official Name and GIIS URL to the new Resource Center details. For more information regarding the site renaming procedure please see: PROC15

Removing a site

Site deletion is not allowed in GOCDB. If a site stops operation, its certification status should be set to "closed". See the section on #Changing_Site_Certification_Status for more information

Changing Site Certification Status

For each site that delivers to the 'Production' Target Infrastructure, GOCDB stores and shows information about its certification status. This reflects the different steps of the official SA1 site certification procedure which typically follows:

The different possible certification statuses are:


Clarifications:


The following site state transitions are allowed:


The following transitions are explicitly forbidden:


Going with the definition of the suspended status, Operations Centre managers have to regularly give their attention to all their suspended sites, so that they are processed within the given maximum time of four months. Sites being in suspended should either be set to closed or brought back in production via the uncertified status.

More information about site certification statuses can be found in SA1 certification and operation procedures documents:

Note: Site certification status cannot be changed by site administrators, and requires intervention of Operations Centre staff.



Defining Pay4Use Properties

Service Endpoints

Definition

A service endpoint is a single entity formed by a hostname, a hosted service and a URL.

GOCDB stores the following information about service endpoints (non exhaustive list):

* The fully qualified hostname of the machine
* The hosted service (see service types below)
* The URL to reach the endpoint
* The IP address of the machine
* The machine's host certificate DN
* A description of the node

As a machine can host many services, there can be many service endpoints per machine.

Example: the machine myhost.domain.org runs a CE, an UI and a UnicoreX service. This will show up in GOCDB as 3 Service Endpoints:

Note that a single host can also specify multiple services of the same service type.

Manipulating service endpoints

Viewing service endpoints

There are different pages in GOCDB where service endpoints are listed:

Each endpoint also has its own listing page. By clicking the link to view a service endpoint, you can see all associated information.

Adding Service Endpoints

There are 2 ways to add new service endpoints to GOCDB, provided you have proper permissions (check the permissions matrix in the #Permissions_associated_to_roles section):

Editing service endpoint information

The editing process will show you the same form as the adding process. To edit a service endpoint, simply click the "edit" link on top of the endpoint's details page.

Removing a service endpoint from a site

to deactivate a service endpoint you have permissions on, simply clic on the "delete" link on top of the endpoint's details page. The interface asks for confirmation before proceeding.

Specific Service Endpoint fields and their impact

"beta" flag (t/f)

This indicates whether the service is a beta service or not (part of the staged rollout process). Beta is the equivalent at service level of the former EGEE Pre-Production Service (PPS)

Host DN

This is the DN of the host certificate for the service. The format of the DN follows that defined by the [OGF Interoperable Certificate Profile] which restricts allowed chars to a PrintableString that does NOT contain characters that cannot be expressed in printable 7-bit ASCII. For a list of allowed chars, see GFD.225.

"production" flag (t/f)

The services Production flag indicates if this service delivers a production quality service to the infrastructure it belongs to (EGI).

"monitoring" flag (t/f)

This flag is taken into account by monitoring tools.

Usage of PRODUCTION and MONITORED flags for EGI Service Endpoints

From 02/12/2014 all production services MUST be monitored (except for emi.ARGUS and VOMS service types).

Production and Monitored

Non-Production and Monitored: YES/NO

Service Groups

A service group is an arbitrary grouping of existing service endpoints that can be distributed across different physical sites and users that belong to the SG (SGs were previously known as 'Virtual Sites'):


NGI Core Services

NGIs can register a number of ‘NGI-Core’ services in GOCDB. A core NGI service is one that is used to calculate the availability and reliability of the NGI. These services fall under the responsibility of the NGI and provide production quality (no testing instances). NGIs can distinguish/flag their core services from their other (non-core) services using one of two ways (see A and B below).

Core Service Requirements

The service instance MUST:

Required Service Types

The following service types are mandatory and all NGIs in the EGI scope should define instances of these services:

Other Mandatory services, depending on middleware deployed by sites under NGI responsibility, are listed here

NGIs should also register their custom core services like accounting, helpdesk if they are registered in GOCDB (for a list of other common core service types see: https://wiki.egi.eu/wiki/NGI_services_in_GOCDB)

Registering NGI Core Services

NGI core services can be grouped/flagged in one of two ways:

It is important that these core service Sites/ServiceGroups adhere to the ‘NGI_XX_SERVICES’ naming scheme. For further details, including a list of existing ‘NGI_XX_SERVICES’ please see: https://wiki.egi.eu/wiki/NGI_services_in_GOCDB

Downtimes

Definition

A downtime is a period of time for which a service is declared to be inoperable. Downtimes may be scheduled (e.g. for software/hardware upgrades), or unscheduled (e.g. power outages). GOCDB stores the following information about downtimes (non exhaustive list):

Manipulating downtimes

Viewing downtimes

There are different pages on which downtimes are listed:

Each downtime has its own listing page, accessible by clicking on the "view" link in downtime listing pages.

Adding downtimes

Provided you have proper permissions (check the permissions matrix in the #Permissions_associated_to_roles section), you can add a downtime by clicking on the Add a Downtime link in the sidebar.

This is done in 2 steps:


Please note:

Editing downtime information

Removing downtimes

To delete a downtime, simply click the delete link on top of the downtime's details page. For integrity reasons, it is only possible to remove downtimes that have not started.

"Good practices" and further understanding

Scheduled or unscheduled?

Depending on the planning of the intervention, downtimes can be:

EGI defines precise rules about what should be declared as scheduled or unscheduled, based on how long in advance the downtime is declared. These rules are described in MAN02#How_to_manage_an_intervention and are enforced as follows:

Notes:


WARNING or OUTAGE?

When declaring a downtime, you will be presented the choice of a "severity", which can be either WARNING or OUTAGE. Please consider the following definitions:

Downtime shortening and extension

Limition rules to downtime extensions are enforced in GOCDB as follows:

Service types

In GOCDB a service type is a technology used to provide a service. Each service endpoint in GOCDB is associated with a service type. Service types are pieces of software while service endpoints are a particular instance of that software running in a certain context.

Service Type Naming Scheme

These service types are used at some grid sites within EGI but aren't EGI operational tools or a part of the core middleware distributions (EMI, gLite, ARC, UNICORE, Globus etc).

Service Type List

To request a new service type, please submit a request for a new service type (described below).

Operational Components (middleware agnostic)


Middleware (ARC, gLite, Unicore)
ARC Middleware

gLite Middleware

* SRM.online: [Site service] Storage Resource Manager for disk only.

Unicore Middleware

Globus Middleware

QosCosGrid (QCG) Middleware

EDGI Middleware (European Desktop Grid Initiative)

Cloud

Other

* ch.cern.cvmfs.stratum.1 Service component (stratum.1) of CernVM file system http://cernvm.cern.ch/portal/filesystem

Custom Service Types
In order to control the proliferation of custom service types, please consider submitting a request for a new service type (described below) before using CUSTOM_SERVICE.

* CUSTOM.pl.plgrid.Bazaar SLA negotiation system between users and resource providers from NGI_PL grid

Adding new services types

Please feel free to make a request for a new service type. For CUSTOM service types, we would like to make this process as light-weight as possible. However, currently all new service type requests need to be assessed by EGI via lightweight review process (by OMB and EGI Ops) so that only suitable types are added to GOCDB and to prevent duplication. Therefore, you can submit your request in one of the following ways:

Note, please provide a suggested SE type name following the naming scheme described above (technology provider's reversed domain . software name) and a brief sentence to describe the service type.

Guidelines here for adding custom service types to SAM for monitoring:

Data Visibility / Scopes

Clear Separation of Concerns

It is important to understand that scopes and Projects are distinct:

EGI Scopes

Reserved Scope Tags

FedCloud Reserved Tag

Elixir Reserved Tag

WLCG Reserved Tags

Extension Properties

NOTE: From GOCDB 5.8 (Autumn/Winter 2016) keys must be unique for a given site, service, or service endpoint, or service group.

Extension Properties in the PI


Examples:


To return all sites that define VO with a value of Alice:

?method=get_site&extensions=(VO=Alice)

Use no value to define a wildcard search, i.e. all sites that define the VO property regardless of value:

?method=get_site&extensions=(VO=)

NOTE: From GOCDB 5.7 (Autumn/Winter 2016) keys must be unique for a given site, service, or service endpoint, or service group. The following section of documentation has not yet been changed to reflect this.

Extensions also supports OR/AND/NOT operators. This can be used to search against multiple key values eg:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)

These can be used together:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)
?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)OR(CPU_HS02_HOUR=2)

When no operator is specified the default is AND, therefore the following:

?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

Is the same as:

?method= get_service_endpoint&extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

The extensions parameter can also be used in conjunction with the existing parameters previously supported:

?method=get_site&extensions=(VO=Alice)NOT(VO=LHCB)&scope=EGI&roc=NGI_UK
?method=get_downtime_nested_services&site_extensions=(eg.2=val.2)&service_extensions=(eg.2=)
?method=get_downtime&site_extensions=(eg.2=val.2)&service_extensions=(eg.2=)

Options for adding a new Project in GocDB

GocDB is multi-tenanted; it can host multiple projects in the same instance. There are a number of different deployment scenarios that can be used to support new projects detailed below. Please contact the GocDB admins/EGI Ops to request the addition of a new project.

1) Add resources (sites/services) to an existing project

2) Add resources (sites/services) to an existing project and add a new Scope tag to represent a sub-grouping

3) Add resources (sites/services) to a new Project and add a new Scope tag to filter by project

How to and FAQ

I get an "error 12227" message when accessing GOC portal with Mozilla/Firefox

This happens when no certificate has been uploaded to your browser. Refer to the "Access to GOCDB" section for more information about GOCDB and X509 certificates.

I am responsible for a site that has recently entered the EGI infrastructure. How do I register it?

Only registered users with an approved role on an NGI can add a new site. If you are the site administrator, the first thing to do is to contact your NGI staff and ask them to add the site for you. Then, register to GOCDB (see the user account section) and ask for a site admin role for your site (see the requesting a role section). Once your role approved, you will be able to edit and change your site information.

Why can't I declare downtimes for my whole site as I used to do in GOCDB3?

For data clarity reasons, it has been decided long ago to only link downtimes to services, thus avoiding the complication of having to check both site and service downtimes to determine whether a service is up or not. The way to declare a downtime for your site is to select all the services of the site in one go when inserting the downtime.

How do I extend a declared schedule downtime?

Because of EGI policies it is not possible to extend a downtime. Recommended good practice for any downtime extension is to declare a new unscheduled downtime, starting just when the frst one finishes. please refere to the downtimes section of this documentation for more information, especially the "downtime extension" paragraph.

I have declared a downtime "at risk", and it turns out to be an outage. How can I declare this properly?

If you have declared the downtime as being at risk and an outage actually happens half way through, you need to update GOCDB to reflect the fact that your site is now down. There is currently no way of doing this by updating the downtime on the fly without having the system considering the whole downtime as being an outage. The best way to proceed is:

How do I switch monitoring on/off for my nodes?

Monitoring status in GOCDB cannot always be switched off. If a node is declared as delivering a production service, rules apply and the node has to be monitored. If you are running a test node and want to switch monitoring off, set both "monitoring" and "production" to "N".

Why nobody has approved my role request yet?

Someone has to approve any request you make, in order to ensure nobody is trying to get inappropriate roles. If yours is not getting approved, this can either be because your request was not legitimate, or most likely because the people that are supposed to do it forgot about it. Please refer to the Roles permissions definitions section of this documentation to determine who should validate your role, and try to get in touch with them. If you are requesting a site admin role, they are likely to be your fellow site admins or your NGI operators.

I am not an EGI user but need access to GOCDB backend to retrieve information for my project. What can I do?

Accessing GOCDB backend through another way than the GOC portal web interface is out of the scope of this documentation. please refer to the technical documentation instead, which is available from  GOCDB Documentation Index.



Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Print/export