Difference between revisions of "GOCDB/Release4/Development/ExtensibilityMechanism"
Line 16: | Line 16: | ||
This extensions parameter allows you to supply an LDAP styled query to further filter your site and endpoint results. For example: | This extensions parameter allows you to supply an LDAP styled query to further filter your site and endpoint results. For example: | ||
?method=get_site&extensions=(VO=Alice) | <pre>?method=get_site&extensions=(VO=Alice)</pre> | ||
Will return all sites that define VO with a value of Alice. A wildcard could also be used: | Will return all sites that define VO with a value of Alice. A wildcard could also be used: | ||
?method=get_site&extensions=(VO=*) | <pre>?method=get_site&extensions=(VO=*)</pre> | ||
Will return all sites that define VO as a key property with any value. Comma separated lists of values are also supported: | Will return all sites that define VO as a key property with any value. Comma separated lists of values are also supported: | ||
?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=1,2,3,4,5) | <pre>?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=1,2,3,4,5)</pre> | ||
This will return any service endpoints that define a CPU_HS01_HOUR with a value of 1, 2, 3, 4, or 5. Extensions also supports OR/AND operators. This can be used to search against multiple key values eg: | This will return any service endpoints that define a CPU_HS01_HOUR with a value of 1, 2, 3, 4, or 5. Extensions also supports OR/AND operators. This can be used to search against multiple key values eg: | ||
?method= get_service_endpoint&extensions=(AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)) | <pre>?method= get_service_endpoint&extensions=(AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))</pre> | ||
This will return endpoints that define both CPU_HS01_HOUR with a value or 1 and CPU_HS02_HOUR with a value of 2. The same query can also be performed with OR: | This will return endpoints that define both CPU_HS01_HOUR with a value or 1 and CPU_HS02_HOUR with a value of 2. The same query can also be performed with OR: | ||
?method= get_service_endpoint&extensions=(OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2)) | <pre>?method= get_service_endpoint&extensions=(OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))</pre> | ||
This can then be used to find sites or service endpoints that all define the same group of key values: | This can then be used to find sites or service endpoints that all define the same group of key values: | ||
?method=get_site&extensions=(AND(VO=Alice,Atlas,CMS,LHCB)) | <pre>?method=get_site&extensions=(AND(VO=Alice,Atlas,CMS,LHCB))</pre> | ||
The extensions parameter can also be used in conjunction with the existing parameters previously supported: | The extensions parameter can also be used in conjunction with the existing parameters previously supported: | ||
?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK | <pre>?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK</pre> | ||
These are currently in testing and can be viewed on the GocDB test server: | These are currently in testing and can be viewed on the GocDB test server: |
Revision as of 19:00, 29 November 2013
Main | EGI.eu operations services | Support | Documentation | Tools | Activities | Performance | Technology | Catch-all Services | Resource Allocation | Security |
GOC DB menu: | Home • | Documentation Index • |
<< Back to GOCDB/Release4/Development
Key-Value Pair Property Bag Extensiblity Mechanism
Introduction
- Original RT request: https://rt.egi.eu/rt/Ticket/Display.html?id=3764
- Simiar to the GLUE2 extensibility mechanim, the core GOCDB entities (NGIs, Sites, Services, ServiceGroups) will be extended so that they can define an optional set of custom key-value pairs.
- In doing this, additional parameters could be supplied to the PI for selecting entities that define a particular set of properties. For example "select all Sites/Services that define the 'MPI=true' property."
- As an extensibility mechanism, we do not intend to impose any restrictions on the values of key-value pairs (other than max value length, e.g. 255 chars ). Key-value pair meaninings should be determined as a community folksonomy.
PI Examples
The URL query could be parameterised with a known GET request paramter (e.g. 'extensions') with a LDAP syntax query string for the value, eg: This extensions parameter allows you to supply an LDAP styled query to further filter your site and endpoint results. For example:
?method=get_site&extensions=(VO=Alice)
Will return all sites that define VO with a value of Alice. A wildcard could also be used:
?method=get_site&extensions=(VO=*)
Will return all sites that define VO as a key property with any value. Comma separated lists of values are also supported:
?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=1,2,3,4,5)
This will return any service endpoints that define a CPU_HS01_HOUR with a value of 1, 2, 3, 4, or 5. Extensions also supports OR/AND operators. This can be used to search against multiple key values eg:
?method= get_service_endpoint&extensions=(AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))
This will return endpoints that define both CPU_HS01_HOUR with a value or 1 and CPU_HS02_HOUR with a value of 2. The same query can also be performed with OR:
?method= get_service_endpoint&extensions=(OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))
This can then be used to find sites or service endpoints that all define the same group of key values:
?method=get_site&extensions=(AND(VO=Alice,Atlas,CMS,LHCB))
The extensions parameter can also be used in conjunction with the existing parameters previously supported:
?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK
These are currently in testing and can be viewed on the GocDB test server:
https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(VO=*)
https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(AND(VO=Alice,LHCB))
Note:
- The extent to which the ldap syntax query string would be supported will have to be explored. A simplified version including only exact pattern matches and simple wildcards may cover most use cases.
- The property bags may only be applied to a subset of entities and PI queries.
Sample PI XML Output
<?xml version="1.0" encoding="UTF-8"?>
<results>
<SERVICE_ENDPOINT PRIMARY_KEY="50257G0">
<PRIMARY_KEY>123456G0</PRIMARY_KEY>
<HOSTNAME>some.host.ac.uk</HOSTNAME>
<GOCDB_PORTAL_URL>https://goc.egi.eu/portal/index.php?Page_Type=View_Object&object_id=123456&grid_id=0</GOCDB_PORTAL_URL>
<HOST_OS>SL5</HOST_OS>
<BETA>N</BETA>
<SERVICE_TYPE>SomeComputeService</SERVICE_TYPE>
<CORE></CORE>
<IN_PRODUCTION>Y</IN_PRODUCTION>
<NODE_MONITORED>Y</NODE_MONITORED>
<SITENAME>DGIREF</SITENAME>
<COUNTRY_NAME>Germany</COUNTRY_NAME>
<COUNTRY_CODE>DE</COUNTRY_CODE>
<ROC_NAME>NGI_DE</ROC_NAME>
<ENDPOINT>
<URL>ldap://ce-cms.vinca.rs:2170/mds-vo-name=AEGIS10-VINCA-CMS,o=grid</URL>
<InterfaceName>RIS</InterfaceName>
</ENDPOINT>
<!-- Key-Value Pair Property Bag attached to SERVICE_ENDPOINT entity -->
<Extensions>
<Extension>
<LocalID>1234</LocalID>
<Key>MPI</Key>
<Value>True</Value>
</Extension>
<Extension>
<LocalID>12345</LocalID>
<Key>somekey</Key>
<Value>valueX</Value>
</Extension>
</Extensions>
</SERVICE_ENDPOINT>
</results>
Potential Use Cases
A number of potential use cases have been reported (https://rt.egi.eu/rt/Ticket/Display.html?id=3764), including:
- Specifying service access protocol.
- Defining custom monitoring parameters.
- Defining custom functionality/capability flags supported by a service (this may be related to MPI in EGI requirements, see: https://documents.egi.eu/public/RetrieveFile?docid=1260&version=3&filename=VT-MPI_Report_v3_final.pdf and https://rt.egi.eu/rt/Ticket/Display.html?id=3396).
Issues
There is the potential for proliferation of key-value pairs that should instead be recorded in fields/properties as part of the data model.