Difference between revisions of "GOCDB/Release4/Development/ExtensibilityMechanism"

From EGIWiki
Jump to: navigation, search
(PI Examples)
(PI Examples)
Line 18: Line 18:
 
==PI Examples==
 
==PI Examples==
 
* The get_sites and get_service_endpoints PI methods will support the 'extensions' URL param. This can be used to filter the Sites and Services by the specified key-value pairs.  
 
* The get_sites and get_service_endpoints PI methods will support the 'extensions' URL param. This can be used to filter the Sites and Services by the specified key-value pairs.  
* Please note, <font color="red"><b>the sytanx for the 'extensions' parameter will be udpated soon and the following examples will no longer work</b></font> once the updates are released!
 
  
 
<pre>?method=get_site&extensions=(VO=Alice)</pre>
 
<pre>?method=get_site&extensions=(VO=Alice)</pre>
Line 26: Line 25:
 
<pre>?method=get_site&extensions=(VO=*)</pre>
 
<pre>?method=get_site&extensions=(VO=*)</pre>
  
<!--
 
Will return all sites that define VO as a key property with any value. Comma separated lists of values are also supported:
 
This will return any services that define a CPU_HS01_HOUR with a value of 1, 2, 3, 4, or 5.
 
<pre>?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=1,2,3,4,5)</pre>
 
-->
 
Extensions also supports OR/AND operators. This can be used to search against multiple key values eg:
 
<pre>?method=get_site&extensions=(AND(VO=Alice)(VO=Emma))</pre>
 
<pre>?method=get_site&extensions=(OR(VO=Alice)(VO=Emma))</pre>
 
<pre>?method= get_service_endpoint&extensions=(AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))</pre>
 
  
The last example will return services that define both CPU_HS01_HOUR with a value or 1 and CPU_HS02_HOUR with a value of 2. The same query can also be performed with OR:
+
Extensions also supports OR/AND/NOT operators. This can be used to search against multiple key values eg:
 +
<pre>?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)</pre>
 +
These can be used together:
 +
<pre>?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)</pre>
 +
<pre>?method= get_service_endpoint&extensions=OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)</pre>
  
<pre>?method= get_service_endpoint&extensions=(OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))</pre>
+
When no operator is supplied in a single the default will be AND, therefore the following:
<!--
+
<pre>?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)</pre>
This can then be used to find sites or service endpoints that all define the same group of key values:
+
Is the same as:
 
+
<pre>?method= get_service_endpoint&extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)</pre>
<pre>?method=get_site&extensions=(AND(VO=Alice,Atlas,CMS,LHCB))</pre>
 
-->
 
  
 
The extensions parameter can also be used in conjunction with the existing parameters previously supported:
 
The extensions parameter can also be used in conjunction with the existing parameters previously supported:
  
 
<pre>?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK</pre>
 
<pre>?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK</pre>
 +
  
 
These are currently in testing and can be viewed on the GocDB test server:
 
These are currently in testing and can be viewed on the GocDB test server:

Revision as of 15:45, 4 December 2013

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


<< Back to GOCDB/Release4/Development

Key-Value Pair Property Bag Extensiblity Mechanism

Introduction

  • Original RT request: https://rt.egi.eu/rt/Ticket/Display.html?id=3764
  • Sites and Services can be extended so that they can define an optional set of custom key-value pairs (akin to GLUE2 ext mech).
  • The 'get_service_endpoint' and 'get_site' PI methods use the ‘extensions’ query parameter to filter sites/services by required key/value pair.
  • This is not yet in production and needs a thorough period of testing first.
  • Bugs are likely at this stage.
  • The extensions parameter only supports basic query syntax (e.g. no nesting of sub-queries and negation).
  • It is probable that the query syntax will change before the final version is released.
  • The results of the get_site and get_service_endpoint PI queries nest new <EXTENSIONS> elements.

PI Examples

  • The get_sites and get_service_endpoints PI methods will support the 'extensions' URL param. This can be used to filter the Sites and Services by the specified key-value pairs.
?method=get_site&extensions=(VO=Alice)

Will return all sites that define VO with a value of Alice. A wildcard could also be used:

?method=get_site&extensions=(VO=*)


Extensions also supports OR/AND/NOT operators. This can be used to search against multiple key values eg:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)(VO=LHCB)

These can be used together:

?method=get_site&extensions=AND(VO=Alice)(VO=Atlas)NOT(VO=LHCB)
?method= get_service_endpoint&extensions=OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

When no operator is supplied in a single the default will be AND, therefore the following:

?method= get_service_endpoint&extensions=(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

Is the same as:

?method= get_service_endpoint&extensions=AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)

The extensions parameter can also be used in conjunction with the existing parameters previously supported:

?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK


These are currently in testing and can be viewed on the GocDB test server:

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(VO=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=AND(VO=Alice,LHCB)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=OR(VO=CMS,LHCB,Atlas,Alice)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=OR(CPU_HS01_HOUR=*)(CPU_HS02_HOUR=5)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=AND(CPU_HS01_HOUR=*)(CPU_HS02_HOUR=5)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)&sitename=RAL-LCG2


Note:

  • The extent to which the ldap syntax query string would be supported will have to be explored. A simplified version including only exact pattern matches and simple wildcards may cover most use cases.
  • The property bags may only be applied to a subset of entities and PI queries.

Sample PI XML Output

<?xml version="1.0" encoding="UTF-8"?>
<results>
	<SITE ID="259" PRIMARY_KEY="458G0" NAME="NGS-HECTOR">
		<PRIMARY_KEY>458G0</PRIMARY_KEY>
		<SHORT_NAME>NGS-HECTOR</SHORT_NAME>
		<OFFICIAL_NAME>HeCTOR, UK National Supercomputing Service
		</OFFICIAL_NAME>
		<SITE_DESCRIPTION>Pending</SITE_DESCRIPTION>
		<GOCDB_PORTAL_URL>
			https://127.0.0.1/DoctrineP/index.php?Page_Type=Siteampid=259
		</GOCDB_PORTAL_URL>
		<HOME_URL>www.hector.ac.uk</HOME_URL>
		<CONTACT_EMAIL>support@hector.ac.uk</CONTACT_EMAIL>
		<CONTACT_TEL>0131-650 5029</CONTACT_TEL>
		<COUNTRY_CODE>GB</COUNTRY_CODE>
		<COUNTRY>United Kingdom</COUNTRY>
		<ROC>NGI_UK</ROC>
		<SUBGRID>NGS</SUBGRID>
		<PRODUCTION_INFRASTRUCTURE>Production</PRODUCTION_INFRASTRUCTURE>
		<CERTIFICATION_STATUS>Uncertified</CERTIFICATION_STATUS>
		<TIMEZONE>UTC</TIMEZONE>
		<CSIRT_EMAIL>helpdesk@hector.ac.uk</CSIRT_EMAIL>
		<DOMAIN>
			<DOMAIN_NAME>hector.ac.uk</DOMAIN_NAME>
		</DOMAIN>
		<EXTENSIONS>
			<EXTENSION>
				<LOCAL_ID>24</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>LHCB</VALUE>
			</EXTENSION>
			<EXTENSION>
				<LOCAL_ID>21</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>Alice</VALUE>
			</EXTENSION>
			<EXTENSION>
				<LOCAL_ID>22</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>Atlas</VALUE>
			</EXTENSION>
			<EXTENSION>
				<LOCAL_ID>23</LOCAL_ID>
				<KEY>VO</KEY>
				<VALUE>CMS</VALUE>
			</EXTENSION>
		</EXTENSIONS>
	</SITE>
</results>

Potential Use Cases

A number of potential use cases have been reported (https://rt.egi.eu/rt/Ticket/Display.html?id=3764), including:

Issues

There is the potential for proliferation of key-value pairs that should instead be recorded in fields/properties as part of the data model.