GOCDB/Release4/Regionalisation

Main

EGI.eu operations services

Support

Documentation

Tools

Activities

Performance

Technology

Catch-all Services

Resource Allocation

Security

Tools menu:	• Main page	• Instructions for developers	• AAI Proxy	• Accounting Portal	• Accounting Repository	• AppDB	• ARGO	• GGUS	• GOCDB
• Message brokers	• Licenses	• OTAGs	• Operations Portal	• Perun	• EGI Collaboration tools	• LToS	• EGI Workload Manager

GOCDB Regionalisation - latest status

The last released version of GOCDB regional module dates from October 2010 (gocdb4.0d7). It is a fully functional tool, and all the basic components are included so that initial deployment can be tested using that version. It is available from https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/tags/gocdb/GOCDB-4.0d7_distrib/RPM/

The code of the latest development version of the module is available from the SVN repository at http://www.sysadmin.hep.ac.uk/svn/grid-monitoring/trunk/gocdb/

The documentation is available on the wiki page GOCDB Regional Module Technical Documentation.

Important note: GOCDB regional modules can be operated in full production but can't synchronise to the central GOCDB.

Regionalisation Plans

Introduction

Steps 1) to 3) below are required for a #3) Regional-Synchronised GOCDB that can publish its 'EGI scoped' data to the Central GOCDB.

1) Data Grouping/Scoping

(Separate EGI and Non-EGI Data)

• Confirmed Development

A GOCDB instance (whether central or regional) needs to differentiate between EGI and non-EGI data. A new group named EGI will be created along with tools to insert / remove data to and from a specified group. With this functionality in place NGIs will be able to store and retrieve both local and EGI level sites in the central GOCDB. (This plan is from "Design A" in https://rt.egi.eu/rt/Ticket/Display.html?id=943#txn-52051).

Notes:

• This will require GUI enhancements, allowing sites to opt-in or opt-out of the EGI group, and new request parameters for restricting the PI queries based on group memberships (e.g. “method=get_site_list_by_group&group=EGI” or “&group=local” or “&group=any”).
• This development is a pre-requisite in order to publish the appropriate data for: #3) Regional-Synchronised GOCDB.

Use Cases:

• Users can centrally include/exclude sites/SEs from the from the whole EGI infrastructure as required (provides extensible data visibility/scoping according to groupings).
• Single point to input/query data for EGI and local/regional data.
• Regional GOCDB installation not required (e.g. for NGIs without effort to install/maintain a regional gocdb).

2) Regional-Standalone GOCDB

• Confirmed Development

We will continue to release a standalone instance of the GOCDB for regional NGI use. This regional instance will be supported regardless of other developments. We will ensure that future developments don't conflict with requirements from NGIs by capturing the NGIs requirements.

Use Cases:

• NGIs can fully customise their own GOCDB as required (e.g. extend DDL/GUI/logic/SQL/whatever)
• Support available centrally

3) Regional-Synchronised GOCDB

• Long Term Development

A regional-standalone GOCDB instance will synchronise is EGI scoped regional data with the central instance.

This would allow NGIs to deploy their own GOCDB, customise it and publish EGI data (constrained by XML schema) to the central portal. Publishing of this data would have to be transactional, most probably via a schema constrained WS/REST interface (preferable over asynch messaging which would require response queues + ACKs). The central portal would be the definitive source for all EGI information.

Notes:

• It may be necessary to have two-way synchronisation, where the central instance would also update regional instances: https://rt.egi.eu/rt/Ticket/Display.html?id=1094.
• NGI extensions and custom data would not be synchronised with the central GOCDB, therefore;
• #1) Data Grouping/Scoping is necessary in order to publish the appropriate data.

Use Cases:

• NGIs can extend their own GOCDB as required (e.g. extend DDL/GUI/logic/SQL)
  • (note, these customizations should extend gocdb, leaving the core tables/queries/schema in tact in order to synchronize the core EGI data, e.g. custom data should be defined in a different table space).
• Single access point to input both EGI and non-EGI data, and query data.
• Central operators would need to override data published by regional GOCDBs (for https://rt.egi.eu/rt/Ticket/Display.html?id=1094)

Current and future releases

• GOCDB4 Central Visualisation Portal
  • last released version: 4.0-4d6 (October 2010)
  • next release: TBD
• GOCDB4 Input System
  • last released version: 4.0-4d6 (October 2010)
  • next production release: TBD

Recent and current developments

The list of GOCDB development items is available in the EGI RT system. The Operational Tools Advisory Group (OTAG) filters and prioritizes these requests. See: [GOCDB RT Requirements Tickets]

Note, dates are estimates.

development item	Estimated release	Estimated date
Provide a production quality package for GOCDB regional module	4.0d7	Oct 2010
improvement of GOCDB failover system and backend replication	4.0.1	March 2011
Consolidation and polishing of the web admin interface for GOCDB regional module	4.0.2	EGI-Y2 and Y3
Provide a SOAP interface to GOCDB in parallel to GOCDBPI	4.0.2	EGI-Y2 and Y3
Work on GOCDB and Operations Portal common front end	?	EGI-Y2 and Y3
GOCDB interface to the dynamic information system (BDII/GluE)	?	EGI-Y3

Regional plans per country/region

Initial plans (as of March 2009)

This page gives an overview of regional plans as discussed and presented during SA1 face to face coordination meeting in Catania, 4 march 2009. See slides available on http://indico.cern.ch/conferenceDisplay.py?confId=51626 for more details on each region/country plans.

NGI plans matrix

• Region 1 means deploying and using a distributed, customisable version of current GOCDB, provided by RAL-STFC
• Region 2 means continuing using central GOCDB
• Region 3 means deploying/using another tool than GOCDB, interfaced with central GOCDB for interoperability purposes

NGI	Region1	Region2	Region3	Timeline	Notes
Asia Pacific ROC	X
Benelux		X
Czech Republic	X			Tests started early 2010
Denmark	X
Finland	X
France	X				Discussions engaged Nov 2009
Germany	X			Tests started Jul 2009
Greece	X				Potentially interested
Ireland	X				Second region 1 pilot
Italy	X
Norway	X
Poland	X
Russia	X
South East Europe ROC			X	Tests started	Will use HGSM - First region 3 pilot
South West Europe ROC			X	Start integration by Feb 2010	Will use HGSM - Second region 3 pilot
Sweden	X
United Kingdom	X			Tests started Jun 2009	First region 1 pilot
Others		X			"Catch-all" usecase for all countries/regions not planning for regionalisation yet.

Specific regional usecases - discussions, requests and ideas

UK NGI

Discussion has first started with NGS. A first summary of GOCDB possible use is available on [NGS wiki] .

• idea from RHUL site: we could have differenciated roles for site admins, e.g. a "primary site admin" which is the one who is contacted when needed. Additional roles can be region specific if we don't need them at a central level
• Current testing of a UK GOCDB module is done at RAL

South East Europe

• SEE ROC plans are to use HGSM as their production regional repository.
• Integration work between HGSM and GOCDB has started. Exchange format is being defined and will be tested during October 2009. Further testing will done in November and December.
• HGSM will be official production repository for SEE by January 2010

Grid-Ireland

Grid-Ireland needs to deploy a standalone GOCDB to be integrated to their certification and test infrastructure.

Italy

• Italy would be in favour of a MySQL version of regional GOCDB.
• Italian model might become hierarchical: one regional GOCDB with 3 "sub-regional" GOCDB below that.

DECH

• DECH ROC (for Germany/Switzerland) has agreed to be the 4th pilot distribution instance
• Current testing of a UK GOCDB module is done at KIT

Serbia

• Serbia would like to try to install and use GOCDB4 as Serbian Grid (AEGIS)information service.

Czech NGI

• The Czech NGI is currently deploying the test regional GOCDB package for evaluation.