Difference between revisions of "GOCDB/PI/Technical Documentation"
m (→Python example) |
m |
||
Line 11: | Line 11: | ||
=== Interface description === | === Interface description === | ||
==== Interface type ==== | ==== Interface type ==== | ||
GOCDB Programmatic Interface is a [http://en.wikipedia.org/wiki/Representational_State_Transfer REST] (Representational State Transfer) based interface over https. The use of https guarantees URLs are properly secured when transiting. Some of the methods are nonetheless public and don't require client side authentication (see [[#Data protection and access]]). | GOCDB Programmatic Interface is a [http://en.wikipedia.org/wiki/Representational_State_Transfer REST] (Representational State Transfer) based interface over https. The use of https guarantees URLs are properly secured when transiting. Some of the methods are nonetheless public and don't require client side authentication (see [[#Data protection and access]]). | ||
==== Entry point ==== | ==== Entry point ==== | ||
Enter GOCDB here https://goc.gridops.org/ or directly at https://goc.gridops.org/portal/ to browse the global EGI information or at https://gocdb4.esc.rl.ac.uk/portal to update information. | Enter GOCDB here https://goc.gridops.org/ or directly at https://goc.gridops.org/portal/ to browse the global EGI information or at https://gocdb4.esc.rl.ac.uk/portal to update information. | ||
Line 20: | Line 23: | ||
==== Outputs ==== | ==== Outputs ==== | ||
Output format is '''XML'''. Calling any implemented method will return you a valid DOM document that can be parsed by any tool on client side. | Output format is '''XML'''. Calling any implemented method will return you a valid DOM document that can be parsed by any tool on client side. | ||
==== Schemas ==== | ==== Schemas ==== | ||
The associated '''XML schema''' for each method is available by calling the method with "&output=xmlschema", e.g. https://goc.gridops.org/gocdbpi/private/?method=get_site&output=xmlschema | The associated '''XML schema''' for each method is available by calling the method with "&output=xmlschema", e.g. https://goc.gridops.org/gocdbpi/private/?method=get_site&output=xmlschema | ||
=== Data protection and access === | === Data protection and access === | ||
==== Protection levels ==== | ==== Protection levels ==== | ||
There are currently 3 protection levels for all methods of the interface | There are currently 3 protection levels for all methods of the interface | ||
Line 38: | Line 42: | ||
Methods at level 1 are available without restrictions. Level 2 requires client to present a valid recognized X509 certificate (reference CA list is the officially agreed EGEE/LCG EUGridPMA list). Level 3 requires client to present a known, registered certificate (certificate DN needs to be stored on GOCDB side for authentication) | Methods at level 1 are available without restrictions. Level 2 requires client to present a valid recognized X509 certificate (reference CA list is the officially agreed EGEE/LCG EUGridPMA list). Level 3 requires client to present a known, registered certificate (certificate DN needs to be stored on GOCDB side for authentication) | ||
==== How to gain access? ==== | |||
If you need to access protected methods, a valid certificate will be enough. If you need to access private methods, please send a mail to [mailto:gocdb-admins@mailtalk.ac.uk gocdb-admins@mailtalk.ac.uk], indicating: | If you need to access protected methods, a valid certificate will be enough. If you need to access private methods, please send a mail to [mailto:gocdb-admins@mailtalk.ac.uk gocdb-admins@mailtalk.ac.uk], indicating: | ||
Line 47: | Line 51: | ||
* An e-mail contact for your application, that will be subscribed to the GOCDB-discussion and announcement mailing list if it is not already there. | * An e-mail contact for your application, that will be subscribed to the GOCDB-discussion and announcement mailing list if it is not already there. | ||
=== Available methods === | |||
{| {{egi-table}} | {| {{egi-table}} | ||
! Method name !! type !! description !! Protection level | ! Method name !! type !! description !! Protection level | ||
Line 89: | Line 92: | ||
=== Client code examples === | === Client code examples === | ||
==== Simple data retrieval using wget ==== | ==== Simple data retrieval using wget ==== | ||
If getting the data locally is what you're after, a simple cron job retrieving data via wget might do the trick. | If getting the data locally is what you're after, a simple cron job retrieving data via wget might do the trick. | ||
Line 106: | Line 111: | ||
'''Note:''' examples are shown using wget-10.2. Some discrepencies are observed between version so check wget help for correct options. Bugs and weird behaviour have also been observed for wget with a version older than 10. | '''Note:''' examples are shown using wget-10.2. Some discrepencies are observed between version so check wget help for correct options. Bugs and weird behaviour have also been observed for wget with a version older than 10. | ||
==== Perl example ==== | |||
Provided by Emir Imamagic, SRCE. This code is used by NCG | Provided by Emir Imamagic, SRCE. This code is used by NCG | ||
Line 122: | Line 127: | ||
'''note:''' Full code is available from [https://twiki.cern.ch/twiki/bin/view/EGEE/EGEESA1SourceRepository EGEE SA1 Source code repository] | '''note:''' Full code is available from [https://twiki.cern.ch/twiki/bin/view/EGEE/EGEESA1SourceRepository EGEE SA1 Source code repository] | ||
==== PHP example ==== | |||
Provided by Guillaume Cessieux, CC-IN2P3. This code is used by ENOC | Provided by Guillaume Cessieux, CC-IN2P3. This code is used by ENOC | ||
Line 130: | Line 135: | ||
* [https://ccenoc.in2p3.fr/GOCDB4/Cgocdb.php?source ENOC interface to GOCDB] | * [https://ccenoc.in2p3.fr/GOCDB4/Cgocdb.php?source ENOC interface to GOCDB] | ||
==== Python example ==== | |||
Provided by Giuseppe Misurelli, CNAF-INFN | Provided by Giuseppe Misurelli, CNAF-INFN | ||
Line 138: | Line 143: | ||
* [http://goc.grid.sinica.edu.tw/gocwiki/GOCDB_Technical_Documentation?action=AttachFile&do=get&target=Get_downtimes_python_example.tar.gz Get_downtimes_python_example.tar.gz] | * [http://goc.grid.sinica.edu.tw/gocwiki/GOCDB_Technical_Documentation?action=AttachFile&do=get&target=Get_downtimes_python_example.tar.gz Get_downtimes_python_example.tar.gz] | ||
=== New requests and improvements === | |||
Requests on GOCDB-PI can take the form of: | Requests on GOCDB-PI can take the form of: | ||
Revision as of 09:59, 15 November 2010
GOCDB Programmatic Interface
Important information about using the GOCDBPI
Releases, changes and announcements
All client tool using GOCDB should register someone to the GOCDB discussion mailing list: gocdb-discuss_at_mailtalk.ac.uk. Releases, new features, changes in methods and general announcements related to this interface are made on this list.
If your application is using the GOCDB Programmatic interface please request to join this mailing list by sending us a mail to gocdb-admins@mailtalk.ac.uk.
Interface description
Interface type
GOCDB Programmatic Interface is a REST (Representational State Transfer) based interface over https. The use of https guarantees URLs are properly secured when transiting. Some of the methods are nonetheless public and don't require client side authentication (see #Data protection and access).
Entry point
Enter GOCDB here https://goc.gridops.org/ or directly at https://goc.gridops.org/portal/ to browse the global EGI information or at https://gocdb4.esc.rl.ac.uk/portal to update information.
Entry point for every single method is given on method definition pages listed below.
Outputs
Output format is XML. Calling any implemented method will return you a valid DOM document that can be parsed by any tool on client side.
Schemas
The associated XML schema for each method is available by calling the method with "&output=xmlschema", e.g. https://goc.gridops.org/gocdbpi/private/?method=get_site&output=xmlschema
Data protection and access
Protection levels
There are currently 3 protection levels for all methods of the interface
- Level 1: public methods (no critical information, no mail, no personal details)
- Level 2: protected methods (contain mails or personal details)
- Level 3: private methods (contains security or critical information)
Methods at level 1 are available without restrictions. Level 2 requires client to present a valid recognized X509 certificate (reference CA list is the officially agreed EGEE/LCG EUGridPMA list). Level 3 requires client to present a known, registered certificate (certificate DN needs to be stored on GOCDB side for authentication)
How to gain access?
If you need to access protected methods, a valid certificate will be enough. If you need to access private methods, please send a mail to gocdb-admins@mailtalk.ac.uk, indicating:
- Your name and affiliation
- The name and purpose of the tool that need to access the data
- certificate DN of the machine(s) that will retrieve the information
- An e-mail contact for your application, that will be subscribed to the GOCDB-discussion and announcement mailing list if it is not already there.
Available methods
Method name | type | description | Protection level |
---|---|---|---|
get_site | read - generic | Returns site information including contacts, grouped by site | 2 |
get_site_list | read - generic | Returns a list of sites with minimal associated information | 1 |
get_site_contacts | read - generic | Returns a list of persons (and associated info) having a role at site level, grouped per site | 2 |
get_site_security_info | read - generic | Returns security contact information for sites | 3 |
get_ngi_list | read - generic | Returns a list of NGIs with minimal associated information | 1 |
get_subgrid_list | read - generic | Returns a list of Subgrids (i.e. registered sub-parts of an NGI) with minimal associated information | 1 |
get_ngi_contacts | read - generic | Returns NGI contact details, including NGI contact mail address and list of NGI staff | 2 |
get_egi_contacts | read - generic | Returns a list of contacts for staff that have a role a EGI level | 2 |
get_downtime | read - generic | Returns a list of EGI downtimes for sites and nodes | 1 |
get_node [unsupported] | read - generic | Returns a list of nodes with associated information and affiliation | 1 |
get_service_endpoint | read - generic | Returns a list of service endpoints (single node x single service) and associated information | 1 |
get_service_types | read - generic | Returns a list of valid service types and associated description | 1 |
get_user | read - generic | Returns a user or a list of users with associated details and roles | 2 |
get_downtime_to_broadcast | read - dedicated | Returns the list of downtimes recently declared with notification settings for CIC portal downtime notification service | 2 |
Note about parameters:
- Unless otherwise stated, all parameters are optional. If no parameter is given, all methods will return results with the wider scope possible.
- It is possible to combine parameters, for example: ?method=get_site&country=Italy&certification_status=Certified&production_status=Production
Client code examples
Simple data retrieval using wget
If getting the data locally is what you're after, a simple cron job retrieving data via wget might do the trick.
- Public methods: You do not need any client certificate, so a simple wget instruction should work:
$ wget --no-check-certificate -O result.xml 'https://goc.gridops.org/gocdbpi/public/?method=get_site_list&sitename=RAL-LCG2'
--no-check-certificate option might be needed to avoid server verification. Note also that URL should be encapsulated into single quotes.
- Protected and private methods: Here you will need authentication, which can be done through wget as shown in next example:
$ wget --certificate=cert.pem --private-key=cert.key.pem --private-key-type=PEM --ca-directory=/etc/grid-security/certificates/ -O result.xml 'https://goc.gridops.org/gocdbpi/private/?method=get_site&sitename=RAL-LCG2'
Note that if you don't specify the --ca-directory option, SSL handshake may fail as server cert cannot be checked.
Note: examples are shown using wget-10.2. Some discrepencies are observed between version so check wget help for correct options. Bugs and weird behaviour have also been observed for wget with a version older than 10.
Perl example
Provided by Emir Imamagic, SRCE. This code is used by NCG
The 3 following examples are GOCDB modules in perl which use LWP::UserAgent for accessing PI and XML::DOM for parsing the data. Data is used to fill internal NCG structure.
Nagios plugin nagios-gocdb-downtime is used for retrieving downtimes and pushing them to Nagios via GridMon::Nagios module:
note: Full code is available from EGEE SA1 Source code repository
PHP example
Provided by Guillaume Cessieux, CC-IN2P3. This code is used by ENOC
The following example shows a simple PHP class with methods used to retrieve and parse XML data from GOCDB-PI using PHP curl and simplexml
Python example
Provided by Giuseppe Misurelli, CNAF-INFN
The following example is a Python client that queries the GOCDB programmatic interface, retrieves info about downtimes for a top entity region using curl, parses xml output with the xml.dom.minidom module and stores data into a MySQl db.
New requests and improvements
Requests on GOCDB-PI can take the form of:
- Change request to an existing method
- Request for new generic methods
- Request for tool specific methods
Such requests are logged in GOCDB task tracker in Savannah. To make a new request, please submit an item in our Savannah wish list.