Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "GOCDB/PI/Technical Documentation"

From EGIWiki
< GOCDB‎ | PI
Jump to navigation Jump to search
Line 268: Line 268:
         org.apache.commons.io.FileUtils.copyURLToFile(url, df);
         org.apache.commons.io.FileUtils.copyURLToFile(url, df);
         System.out.println("downloadFile: "+df.getAbsolutePath());
         System.out.println("downloadFile: "+df.getAbsolutePath());
        String version = System.getProperty("java.version");
        System.out.println("Using Java version: "+version);
     }
     }
}
}

Revision as of 13:36, 11 March 2014

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


GOC DB menu: Home Documentation Index


GOCDB Programmatic Interface

Important information about using the GOCDBPI

Releases, changes and announcements

All client tool using GOCDB should subscribe to the GOCDB discussion mailing list: gocdb-discuss_at_mailman.egi.eu. Releases, new features, changes in methods and general announcements related to this interface are made on this list.

If your application is using the GOCDB Programmatic interface please subscribe to gocdb-discuss or send your subscription request by mail to gocdb-admins_at_mailman.egi.eu.

Interface description

Interface type

GOCDB Programmatic Interface is a REST (Representational State Transfer) based interface over https. The use of https guarantees URLs are properly secured when transiting. Some of the methods are nonetheless public and don't require client side authentication (see #Data protection and access).

Entry point

Enter GOCDB here https://goc.egi.eu/ or directly at https://goc.egi.eu/portal/ to browse and edit the global EGI information. Entry point for every single method is given on method definition pages listed below.

Outputs

Output format is XML. Calling any implemented method will return you a valid DOM document that can be parsed by any tool on client side.

Data protection and access

Protection levels

There are currently 3 protection levels for all methods of the interface

  • Level 1: public methods (no critical information, no mail, no personal details)
  • Level 2: protected methods (contain mails or personal details)
  • Level 3: private methods (contains security or critical information)

Methods at level 1 are available without restrictions. Level 2 requires client to present a valid recognized X509 certificate (reference CA list is the officially agreed EGI/LCG EUGridPMA list). Level 3 requires client to present a known, registered certificate (certificate DN needs to be stored on GOCDB side for authentication)

How to gain access?

If you need to access protected methods, a valid certificate will be enough. If you need to access private methods, please send a mail to gocdb-admins_at_mailman.egi.eu, indicating:

  • Your name and affiliation
  • The name and purpose of the tool that need to access the data
  • certificate DN of the machine(s) that will retrieve the information
  • An e-mail contact for your application, that will be subscribed to the GOCDB-discussion and announcement mailing list if it is not already there.

Data Visiblity

Sites and service endpoints can be shown or hidden from EGI and this affects what data is shown through the PI. By default the PI methods will return only objects that are marked as visible to EGI. For example get_site will, by default, return only sites that are visible to EGI. Similarly, get_downtime will only show downtimes that are linked to EGI service endpoints.

The "scope=Local" parameter can be used to access non-EGI data through the PI. For example: https://goc.egi.eu/gocdbpi/private/?method=get_site&scope=Local

Our user facing documentation for this feature is available at GOCDB/Input System User Documentation#Data_Visibility.

Available methods

Method name type description Protection level
get_site read - generic Returns site information including contacts, grouped by site 2
get_site_list read - generic Returns a list of sites with minimal associated information 1
get_site_contacts read - generic Returns a list of persons (and associated info) having a role at site level, grouped per site 2
get_site_security_info read - generic Returns security contact information for sites 3
get_roc_list read - generic Returns a list of NGIs with minimal associated information 1
get_subgrid_list read - generic Returns a list of Subgrids (i.e. registered sub-parts of an NGI) with minimal associated information 1
get_roc_contacts read - generic Returns NGI contact details, including NGI contact mail address and list of NGI staff 2
get_egee_contacts read - generic Returns a list of contacts for staff that have a role a EGI level 2
get_downtime read - generic Returns a list of service downtimes (downtimes are repeated for each affected service) 1
get_downtime_nested_services read - generic Returns a list of downtimes with affected services nested as child elements of the downtime 1
get_service_endpoint read - generic Returns a list of service endpoints (single node x single service) and associated information 1
get_service_types read - generic Returns a list of valid service types and associated description 1
get_user read - generic Returns a user or a list of users with associated details and roles 2
get_downtime_to_broadcast read - dedicated Returns the list of downtimes recently declared with notification settings for CIC portal downtime notification service 2
get_cert_status_changes read - generic Returns a list of changes to certification statuses 2
get_cert_status_date read - generic Returns a list of current certification statuses for Production sites and the date they entered that status 2
get_service_group read - generic Returns a list of service groups and the service endpoints under those groups. 2
get_service_group_role read - generic Returns a list of service groups and the roles held by users over these groups. 2
get_ngi read - generic Returns a list of NGIs with contact information 2
get_project_contacts read - generic Returns a list of persons (and associated info) having a role over a project. 2

Note about parameters:

  • Unless otherwise stated, all parameters are optional. If no parameter is given, all methods will return results with the wider scope possible.
  • It is possible to combine parameters, for example: ?method=get_site&country=Italy&certification_status=Certified&production_status=Production

Client code examples

Simple data retrieval using wget/curl

If getting the data locally is what you're after, a simple cron job retrieving data via wget might do the trick.

  • Public methods: You do not need any client certificate, so a simple wget instruction should work:
$ wget --no-check-certificate -O result.xml 'https://goc.egi.eu/gocdbpi/public/?method=get_site_list&sitename=RAL-LCG2'

--no-check-certificate option might be needed to avoid server verification. Note also that URL should be encapsulated into single quotes.

  • Protected and private methods: Here you will need authentication, which can be done through wget as shown in next example:
$ wget --certificate=cert.pem --private-key=cert.key.pem --private-key-type=PEM --ca-directory=/etc/grid-security/certificates/ -O result.xml 'https://goc.egi.eu/gocdbpi/private/?method=get_site&sitename=RAL-LCG2'

Note that if you don't specify the --ca-directory option, SSL handshake may fail as server cert cannot be checked.

Note: examples above are shown using wget-10.2. Some discrepencies are observed between version so check wget help for correct options. Bugs and weird behaviour have also been observed for wget with a version older than 10 and using wget on cygwin.

$ curl -k https://goc.dl.ac.uk/gocdbpi/private/?method=get_site_security_info --cert usercert.pem  --key userkey.pem

Perl example

Provided by Emir Imamagic, SRCE. This code is used by NCG

The 3 following examples are GOCDB modules in perl which use LWP::UserAgent for accessing PI and XML::DOM for parsing the data. Data is used to fill internal NCG structure.

Nagios plugin nagios-gocdb-downtime is used for retrieving downtimes and pushing them to Nagios via GridMon::Nagios module:

note Full code is available from cern svn

PHP example

Provided by Guillaume Cessieux, CC-IN2P3. This code is used by ENOC

The following example shows a simple PHP class with methods used to retrieve and parse XML data from GOCDB-PI using PHP curl and simplexml


Python example

Provided by Giuseppe Misurelli, CNAF-INFN

The following example is a Python client that queries the GOCDB programmatic interface, retrieves info about downtimes for a top entity region using curl, parses xml output with the xml.dom.minidom module and stores data into a MySQl db.


Java example

Provided by David Meredith, STFC

import java.io.File;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import org.junit.Test;

public class DownloadFileSample {
    // trust all certificates (don't do this is production) 
    TrustManager[] trustAllCerts = new TrustManager[]{
        new X509TrustManager() {
            public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                return null;
            }
            public void checkClientTrusted(
                    java.security.cert.X509Certificate[] certs, String authType) {
            }
            public void checkServerTrusted(
                    java.security.cert.X509Certificate[] certs, String authType) {
            }
        }
    };

    @Test 
    public void testIt() throws Exception{
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
        URL url = new URL("https://goc.egi.eu/gocdbpi/public/?method=get_site_list"); 
        File df = File.createTempFile("sampleSiteList", ".xml");
        org.apache.commons.io.FileUtils.copyURLToFile(url, df);
        System.out.println("downloadFile: "+df.getAbsolutePath());
        String version = System.getProperty("java.version");
        System.out.println("Using Java version: "+version);
    }
}

New requests and improvements

Requests on GOCDB-PI can take the form of:

  • Change request to an existing method
  • Request for new generic methods
  • Request for tool specific methods

Such requests are logged in The GOCDB RT Requirements Queue Tickets: View and submit RT Requirements tickets specifically for GOCDB