Difference between revisions of "GOCDB/Release4/Development/ExtensibilityMechanism"

From EGIWiki
Jump to: navigation, search
(Introduction)
(PI Examples)
Line 65: Line 65:
 
https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)&sitename=RAL-LCG2
 
https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)&sitename=RAL-LCG2
  
 +
https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site_list&extensions=AND(VO=Alice)(VO=CMS)
 +
 +
https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site_list&extensions=(VO=LHCB)NOT(VO=Alice)(VO=Atlas)(VO=CMS)
  
 
Note:
 
Note:

Revision as of 10:41, 5 December 2013

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


<< Back to GOCDB/Release4/Development

Key-Value Pair Property Bag Extensiblity Mechanism

Introduction

  • Original RT request: https://rt.egi.eu/rt/Ticket/Display.html?id=3764
  • Sites and Services can be extended so that they can define an optional set of custom key-value pairs (akin to GLUE2 ext mech).
  • The 'get_service_endpoint', 'get_site' and 'get_site_list' PI methods use the ‘extensions’ query parameter to filter sites/services by required key/value pair.
  • This is not yet in production and needs a thorough period of testing first.
  • Bugs are likely at this stage.
  • The extensions parameter only supports basic query syntax (e.g. no nesting of sub-queries).
  • The results of the get_site and get_service_endpoint PI queries nest new <EXTENSIONS> elements.
  • The format of the 'extensions' expression is one or more (key=value) pairs enclosed in brackets. (K=v) pairs can be optionally prefixed with one of following operators: AND, OR, NOT. If no operator is specified before the first (k=v) pair, then AND is assumed. The operator applies to ALL (k=v) pairs to the right of the operator until another operator is encountered, for example:


AND(key1=val)(key2=va2)OR(key3=val3)(key4=val4)NOT(key5=val5)(key6=val6)
is equivalent to: (note no leading AND which is assumed)
(key1=val)(key2=va2)OR(key3=val3)(key4=val4)NOT(key5=val5)(key6=val6)

PI Examples

  • The get_sites and get_service_endpoints PI methods will support the 'extensions' URL param. This can be used to filter the Sites and Services by the specified key-value pairs.
?method=get_site&extensions=(VO=Alice)

Will return all sites that define VO with a value of Alice. A wildcard could also be used:

?method=get_site&extensions=(VO=*)

Extensions also supports OR/AND/NOT operators. This can be used to search against multiple key values eg:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)

These can be used together:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)
?method= get_service_endpoint&extensions=OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

When no operator is the default will be AND, therefore the following:

?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

Is the same as:

?method= get_service_endpoint&extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

The extensions parameter can also be used in conjunction with the existing parameters previously supported:

?method=get_site&extensions=OR(VO=Alice)NOT(VO=LHCB)&scope=EGI&roc=NGI_UK


These are currently in testing and can be viewed on the GocDB test server:

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(VO=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=AND(VO=Alice)(VO=CMS)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(VO=LHCB)NOT(VO=Alice)(VO=Atlas)(VO=CMS)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=OR(CPU_HS01_HOUR=*)NOT(CPU_HS02_HOUR=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=AND(CPU_HS01_HOUR=*)(CPU_HS02_HOUR=5)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=OR(CPU_HS01_HOUR=*)(CPU_HS02_HOUR=5)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)&sitename=RAL-LCG2

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site_list&extensions=AND(VO=Alice)(VO=CMS)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site_list&extensions=(VO=LHCB)NOT(VO=Alice)(VO=Atlas)(VO=CMS)

Note:

  • The extent to which the ldap syntax query string would be supported will have to be explored. A simplified version including only exact pattern matches and simple wildcards may cover most use cases.
  • The property bags may only be applied to a subset of entities and PI queries.

Sample PI XML Output

<?xml version="1.0" encoding="UTF-8"?>
<results>
	<SITE ID="259" PRIMARY_KEY="458G0" NAME="NGS-HECTOR">
		<PRIMARY_KEY>458G0</PRIMARY_KEY>
		<SHORT_NAME>NGS-HECTOR</SHORT_NAME>
		<OFFICIAL_NAME>HeCTOR, UK National Supercomputing Service
		</OFFICIAL_NAME>
		<SITE_DESCRIPTION>Pending</SITE_DESCRIPTION>
		<GOCDB_PORTAL_URL>
			https://127.0.0.1/DoctrineP/index.php?Page_Type=Siteampid=259
		</GOCDB_PORTAL_URL>
		<HOME_URL>www.hector.ac.uk</HOME_URL>
		<CONTACT_EMAIL>support@hector.ac.uk</CONTACT_EMAIL>
		<CONTACT_TEL>0131-650 5029</CONTACT_TEL>
		<COUNTRY_CODE>GB</COUNTRY_CODE>
		<COUNTRY>United Kingdom</COUNTRY>
		<ROC>NGI_UK</ROC>
		<SUBGRID>NGS</SUBGRID>
		<PRODUCTION_INFRASTRUCTURE>Production</PRODUCTION_INFRASTRUCTURE>
		<CERTIFICATION_STATUS>Uncertified</CERTIFICATION_STATUS>
		<TIMEZONE>UTC</TIMEZONE>
		<CSIRT_EMAIL>helpdesk@hector.ac.uk</CSIRT_EMAIL>
		<DOMAIN>
			<DOMAIN_NAME>hector.ac.uk</DOMAIN_NAME>
		</DOMAIN>
		<EXTENSIONS>
			<EXTENSION>
				<LOCAL_ID>24</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>LHCB</VALUE>
			</EXTENSION>
			<EXTENSION>
				<LOCAL_ID>21</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>Alice</VALUE>
			</EXTENSION>
			<EXTENSION>
				<LOCAL_ID>22</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>Atlas</VALUE>
			</EXTENSION>
			<EXTENSION>
				<LOCAL_ID>23</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>CMS</VALUE>
			</EXTENSION>
		</EXTENSIONS>
	</SITE>
</results>

Potential Use Cases

A number of potential use cases have been reported (https://rt.egi.eu/rt/Ticket/Display.html?id=3764), including:

Issues

There is the potential for proliferation of key-value pairs that should instead be recorded in fields/properties as part of the data model.