Difference between revisions of "GOCDB/Release4/Development/ExtensibilityMechanism"

From EGIWiki
Jump to: navigation, search
(PI Examples)
(Sample PI XML Output)
Line 67: Line 67:
 
<?xml version="1.0" encoding="UTF-8"?>
 
<?xml version="1.0" encoding="UTF-8"?>
 
<results>
 
<results>
   <SERVICE_ENDPOINT PRIMARY_KEY="50257G0">
+
   <results>
      <PRIMARY_KEY>123456G0</PRIMARY_KEY>
+
<SITE ID="259" PRIMARY_KEY="458G0" NAME="NGS-HECTOR">
      <HOSTNAME>some.host.ac.uk</HOSTNAME>
+
<PRIMARY_KEY>458G0</PRIMARY_KEY>
      <GOCDB_PORTAL_URL>https://goc.egi.eu/portal/index.php?Page_Type=View_Object&amp;object_id=123456&amp;grid_id=0</GOCDB_PORTAL_URL>
+
<SHORT_NAME>NGS-HECTOR</SHORT_NAME>
      <HOST_OS>SL5</HOST_OS>
+
<OFFICIAL_NAME>HeCTOR, UK National Supercomputing Service</OFFICIAL_NAME>
      <BETA>N</BETA>
+
<SITE_DESCRIPTION>Pending</SITE_DESCRIPTION>
      <SERVICE_TYPE>SomeComputeService</SERVICE_TYPE>
+
<GOCDB_PORTAL_URL>
      <CORE></CORE>
+
https://127.0.0.1/DoctrineP/index.php?Page_Type=Site&id=259
      <IN_PRODUCTION>Y</IN_PRODUCTION>
+
</GOCDB_PORTAL_URL>
      <NODE_MONITORED>Y</NODE_MONITORED>
+
<HOME_URL>www.hector.ac.uk</HOME_URL>
      <SITENAME>DGIREF</SITENAME>
+
<CONTACT_EMAIL>support@hector.ac.uk</CONTACT_EMAIL>
      <COUNTRY_NAME>Germany</COUNTRY_NAME>
+
<CONTACT_TEL>0131-650 5029</CONTACT_TEL>
      <COUNTRY_CODE>DE</COUNTRY_CODE>
+
<COUNTRY_CODE>GB</COUNTRY_CODE>
      <ROC_NAME>NGI_DE</ROC_NAME>
+
<COUNTRY>United Kingdom</COUNTRY>
      <ENDPOINT>
+
<ROC>NGI_UK</ROC>
        <URL>ldap://ce-cms.vinca.rs:2170/mds-vo-name=AEGIS10-VINCA-CMS,o=grid</URL>
+
<SUBGRID>NGS</SUBGRID>
        <InterfaceName>RIS</InterfaceName>
+
<PRODUCTION_INFRASTRUCTURE>Production</PRODUCTION_INFRASTRUCTURE>
      </ENDPOINT>  
+
<CERTIFICATION_STATUS>Uncertified</CERTIFICATION_STATUS>
      <!-- Key-Value Pair Property Bag attached to SERVICE_ENDPOINT entity -->
+
<TIMEZONE>UTC</TIMEZONE>
      <Extensions>
+
<CSIRT_EMAIL>helpdesk@hector.ac.uk</CSIRT_EMAIL>
            <Extension>
+
<DOMAIN>
                <LocalID>1234</LocalID>
+
<DOMAIN_NAME>hector.ac.uk</DOMAIN_NAME>
                <Key>MPI</Key>
+
</DOMAIN>
                <Value>True</Value>  
+
<EXTENSIONS>
            </Extension>
+
<EXTENSION>
            <Extension>
+
<LOCAL_ID>24</LOCAL_ID>
                <LocalID>12345</LocalID>
+
<KEY>VO</KEY>
                <Key>somekey</Key>
+
<VALUE>LHCB</VALUE>
                <Value>valueX</Value>  
+
</EXTENSION>
            </Extension>
+
<EXTENSION>
      </Extensions>
+
<LOCAL_ID>21</LOCAL_ID>
  </SERVICE_ENDPOINT>
+
<KEY>VO</KEY>
 +
<VALUE>Alice</VALUE>
 +
</EXTENSION>
 +
<EXTENSION>
 +
<LOCAL_ID>22</LOCAL_ID>
 +
<KEY>VO</KEY>
 +
<VALUE>Atlas</VALUE>
 +
</EXTENSION>
 +
<EXTENSION>
 +
<LOCAL_ID>23</LOCAL_ID>
 +
<KEY>VO</KEY>
 +
<VALUE>CMS</VALUE>
 +
</EXTENSION>
 +
</EXTENSIONS>
 +
</SITE>
  
 
</results>
 
</results>

Revision as of 18:00, 29 November 2013

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


<< Back to GOCDB/Release4/Development

Key-Value Pair Property Bag Extensiblity Mechanism

Introduction

  • Original RT request: https://rt.egi.eu/rt/Ticket/Display.html?id=3764
  • Simiar to the GLUE2 extensibility mechanim, the core GOCDB entities (NGIs, Sites, Services, ServiceGroups) will be extended so that they can define an optional set of custom key-value pairs.
  • In doing this, additional parameters could be supplied to the PI for selecting entities that define a particular set of properties. For example "select all Sites/Services that define the 'MPI=true' property."
  • As an extensibility mechanism, we do not intend to impose any restrictions on the values of key-value pairs (other than max value length, e.g. 255 chars ). Key-value pair meaninings should be determined as a community folksonomy.

PI Examples

The URL query could be parameterised with a known GET request paramter (e.g. 'extensions') with a LDAP syntax query string for the value, eg: This extensions parameter allows you to supply an LDAP styled query to further filter your site and endpoint results. For example:

?method=get_site&extensions=(VO=Alice)

Will return all sites that define VO with a value of Alice. A wildcard could also be used:

?method=get_site&extensions=(VO=*)

Will return all sites that define VO as a key property with any value. Comma separated lists of values are also supported:

?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=1,2,3,4,5)

This will return any service endpoints that define a CPU_HS01_HOUR with a value of 1, 2, 3, 4, or 5. Extensions also supports OR/AND operators. This can be used to search against multiple key values eg:

?method= get_service_endpoint&extensions=(AND(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))

This will return endpoints that define both CPU_HS01_HOUR with a value or 1 and CPU_HS02_HOUR with a value of 2. The same query can also be performed with OR:

?method= get_service_endpoint&extensions=(OR(CPU_HS01_HOUR=1)(CPU_HS02_HOUR=2))

This can then be used to find sites or service endpoints that all define the same group of key values:

?method=get_site&extensions=(AND(VO=Alice,Atlas,CMS,LHCB))

The extensions parameter can also be used in conjunction with the existing parameters previously supported:

?method=get_site&extensions=(OR(VO=Alice,Atlas,CMS,LHCB))&scope=EGI&roc=NGI_UK

These are currently in testing and can be viewed on the GocDB test server:

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(VO=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(AND(VO=Alice,LHCB))

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_site&extensions=(OR(VO=CMS,LHCB,Atlas,Alice))

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(OR(CPU_HS01_HOUR=*)(CPU_HS02_HOUR=5))

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(AND(CPU_HS01_HOUR=*)(CPU_HS02_HOUR=5))

https://gocdb-test.esc.rl.ac.uk/v5_pi/public/?method=get_service_endpoint&extensions=(CPU_HS01_HOUR=*)&sitename=RAL-LCG2


Note:

  • The extent to which the ldap syntax query string would be supported will have to be explored. A simplified version including only exact pattern matches and simple wildcards may cover most use cases.
  • The property bags may only be applied to a subset of entities and PI queries.

Sample PI XML Output

<?xml version="1.0" encoding="UTF-8"?>
<results>
   <results>
<SITE ID="259" PRIMARY_KEY="458G0" NAME="NGS-HECTOR">
<PRIMARY_KEY>458G0</PRIMARY_KEY>
<SHORT_NAME>NGS-HECTOR</SHORT_NAME>
<OFFICIAL_NAME>HeCTOR, UK National Supercomputing Service</OFFICIAL_NAME>
<SITE_DESCRIPTION>Pending</SITE_DESCRIPTION>
<GOCDB_PORTAL_URL>
https://127.0.0.1/DoctrineP/index.php?Page_Type=Site&id=259
</GOCDB_PORTAL_URL>
<HOME_URL>www.hector.ac.uk</HOME_URL>
<CONTACT_EMAIL>support@hector.ac.uk</CONTACT_EMAIL>
<CONTACT_TEL>0131-650 5029</CONTACT_TEL>
<COUNTRY_CODE>GB</COUNTRY_CODE>
<COUNTRY>United Kingdom</COUNTRY>
<ROC>NGI_UK</ROC>
<SUBGRID>NGS</SUBGRID>
<PRODUCTION_INFRASTRUCTURE>Production</PRODUCTION_INFRASTRUCTURE>
<CERTIFICATION_STATUS>Uncertified</CERTIFICATION_STATUS>
<TIMEZONE>UTC</TIMEZONE>
<CSIRT_EMAIL>helpdesk@hector.ac.uk</CSIRT_EMAIL>
<DOMAIN>
<DOMAIN_NAME>hector.ac.uk</DOMAIN_NAME>
</DOMAIN>
<EXTENSIONS>
<EXTENSION>
<LOCAL_ID>24</LOCAL_ID>
<KEY>VO</KEY>
<VALUE>LHCB</VALUE>
</EXTENSION>
<EXTENSION>
<LOCAL_ID>21</LOCAL_ID>
<KEY>VO</KEY>
<VALUE>Alice</VALUE>
</EXTENSION>
<EXTENSION>
<LOCAL_ID>22</LOCAL_ID>
<KEY>VO</KEY>
<VALUE>Atlas</VALUE>
</EXTENSION>
<EXTENSION>
<LOCAL_ID>23</LOCAL_ID>
<KEY>VO</KEY>
<VALUE>CMS</VALUE>
</EXTENSION>
</EXTENSIONS>
</SITE>

</results>

Potential Use Cases

A number of potential use cases have been reported (https://rt.egi.eu/rt/Ticket/Display.html?id=3764), including:

Issues

There is the potential for proliferation of key-value pairs that should instead be recorded in fields/properties as part of the data model.