Adding Custom Service to Information System

Introduction

Grid information systems are mission-critical components in today's production grid infrastructures. They provide detailed information about grid services which is needed for various different tasks. The EMI information system has a hierarchical structure of three levels. The fundamental building block used in this hierarchy is the Berkley Database Information Index (BDII). Although the BDII has additional complexity, it can be visualized as an LDAP database. The resource level or core BDII is usually co-located with the grid service and provides information about that service. Each grid site runs a site level BDII. This aggregates the information from all the resource level BDIIs running at that site. The top level BDII aggregates all the information from all the site level BDIIs and hence contains information about all grid services. There are multiple instances of the top level BDII in order to provide a fault tolerant, load balanced service. The information system clients query a top level BDII to find the information that they require.

The Freedom of Choice for Resources mechanism (FCR) is used within a top level BDII to enable the Virtual Organization (VO) to influence their usage of specific services. The FCR portal generates a list of services available to a VO. The portal can be used by the VO manager to either white list or black list sites. This information is then downloaded by the top level BDII and it deletes the ACL for the VO on that specific service from the database. This results in the service not being matched in a query from that VO. The information system is bootstrapped from the information in the Grid Operations Center Database (GOC DB). When a site registers, it enters the URL for the site level BDII into the GOC DB. The GOC DB generates a list of LDAP URLs for all the sites in the grid and this is downloaded by the information provider running on the top level BDII. These URLs are then used to query all the site level BDII and the result is used to populate the top level BDII.

BDII architecture

What is Grid Information System ? (Reference)

Catch-All Top-BDII (Information System Service)

Process

Service requestor
1. Decide on a name for the service type (and endpoint type(s) as appropriate). First see if it fits into one of the existing types; if not, consider whether it should also be aligned with the GOC DB. Names should normally follow a "reverse DNS" style based on a suitable project or organisation name, e.g. org.glite.voms, unless they are "well-known" names like http. For names which are also used in the GOC DB follow the standard procedure for that; otherwise notify the new name(s) to the OGF GLUE working group, or ask EGI.eu to do it.
2. Decide what information to publish. There is a fairly small number of attributes which must be published, but potentially anything can be published although it must be fitted into the GLUE schema structure. Information can be published either in the GLUE 1 or GLUE 2 format or both - GLUE 2 is now recommended. It is potentially possible to extend the GLUE 2 schema, but this would be a major project so it should be avoided if possible.
3. For each piece of information to be published, find a way to collect the information from a script running on a service node. Depending on the nature of the information this may be hard-wired, read from a configuration file, or read dynamically in some way. The information must be collected quickly, within about a second at most, and in a way which is normally guaranteed to succeed. If this is not the case, e.g. if a remote service needs to be contacted, some level of indirection should be used, e.g. a cron job which caches the information in a file to be read by the information provider.
4. Write one or more information providers to output the information according to the schema structure in the LDIF format. For simple services a generic information provider is available as part of the BDII core package; this just requires a configuration file which, for each attribute to be published, specifies a command which writes the value to stdout, so there is no need to understand the schema structure in detail. However, for more complex cases a dedicated provider may be required.
5. Add the BDII core package to the software installed on the service node, install and configure the information providers as necessary and add scripts and/or LDIF files to the relevant directories as described in the BDII documentation. Then start the bdii service. This should result in the information being published from the node.
6. For information to appear in the overall information system ("top-level BDIIs") the service node should be added to the configuration of a site BDII at some suitable Grid site. It is not necessary for the service node to be physically located at that site as long as the site BDII has sufficient access to read the information. If the service is registered in the GOC DB it should be registered under the same site.