Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Accounting Portal API"

From EGIWiki
Jump to navigation Jump to search
Line 47: Line 47:


{| class="wikitable"
{| class="wikitable"
!Path
!View
!Extra Parameters
!Data shown
!Data shown
|-
|-
Line 56: Line 55:
|-
|-
|report/disciplines/
|report/disciplines/
|Infrastructure, discipline list
|Discipline report
|Discipline report
|-
|-
|wlcg/report/tier2/
|wlcg/report/tier2/
|Day/hour
|WLCG Tier2 Report
|WLCG Tier2 Report
|-
|-
|wlcg/report/countries/
|wlcg/report/countries/
|Day/hour
|WLCG Countries Report
|WLCG Countries Report
|-
|-
|wlcg/report/tier1/
|wlcg/report/tier1/
|N/A
|WLCG Tier1 Report
|WLCG Tier1 Report
|-
|-
|report/interngi/
|report/interngi/
|No
|InterNGI Report
|InterNGI Report
|-
|-
|report/vomet/
|report/vomet/
|No
|VO activity report
|VO activity report
|-
|-
|disciplines/cloud/
|disciplines/cloud/
|No
|Cloud disciplines overview
|Cloud disciplines overview
|-
|-
|disciplines/cloud/
|disciplines/cloud/
|Discipline
|Cloud Discipline detail
|Cloud Discipline detail
|-
|-
|disciplines/
|disciplines/
|No
|HTC disciplines overview
|HTC disciplines overview
|-
|-
|disciplines/
|disciplines/
|Discipline
|HTC Discipline detail
|HTC Discipline detail
|-
|-
|/tier1/cloud/
|/tier1/cloud/
|No
|WLCG Cloud Tier1 overview
|WLCG Cloud Tier1 overview
|-
|-
|/tier1/cloud/node/
|/tier1/cloud/node/
|Node
|WLCG Cloud Tier1 node detail
|WLCG Cloud Tier1 node detail
|-
|-
|/tier1/cloud/site/
|/tier1/cloud/site/
|Site
|WLCG Cloud Tier1 site detail
|WLCG Cloud Tier1 site detail
|-
|-
|/tier1/
|/tier1/
|No
|WLCG HTC Tier1 overview
|WLCG HTC Tier1 overview
|-
|-
|/tier1/node/
|/tier1/node/
|Node
|WLCG HTC Tier1 node detail
|WLCG HTC Tier1 node detail
|-
|-
|/tier1/site/
|/tier1/site/
|Site
|WLCG HTC Tier1 site detail
|WLCG HTC Tier1 site detail
|-
|-
|/tier2/cloud/
|/tier2/cloud/
|No
|WLCG Cloud Tier2 overview
|WLCG Cloud Tier2 overview
|-
|-
|/tier2/cloud/federation/
|/tier2/cloud/federation/
|Federation
|WLCG Cloud Tier2 federation detail
|WLCG Cloud Tier2 federation detail
|-
|-
|/tier2/cloud/country/
|/tier2/cloud/country/
|Country
|WLCG Cloud Tier2 country detail
|WLCG Cloud Tier2 country detail
|-
|-
|/tier2/cloud/site/
|/tier2/cloud/site/
|Site
|WLCG Cloud Tier2 site detail
|WLCG Cloud Tier2 site detail
|-
|-
|/tier2/
|/tier2/
|No
|WLCG HTC Tier2 overview
|WLCG HTC Tier2 overview
|-
|-
|/tier2/federation/
|/tier2/federation/
|Federation
|WLCG HTC Tier2 federation detail
|WLCG HTC Tier2 federation detail
|-
|-
|/tier2/country/
|/tier2/country/
|Country
|WLCG HTC Tier2 country detail
|WLCG HTC Tier2 country detail
|-
|-
|/tier2/site/
|/tier2/site/
|Site
|WLCG HTC Tier2 site detail
|WLCG HTC Tier2 site detail
|-
|-
|/egi/
|[[Accounting_Portal_API/egi/htc/operations_centres/|/egi/htc/operations_centres/]]
|No
|EGI HTC NGI overview
|EGI HTC NGI overview
|-
|-
|/egi/ngi/
|/egi/ngi/
|NGI (Operations Centre)
|EGI HTC NGI detail
|EGI HTC NGI detail
|-
|-
|/egi/country/
|/egi/country/
|Country
|EGI HTC Country detail
|EGI HTC Country detail
|-
|-
|/egi/countries/
|/egi/countries/
|No
|EGI HTC Country overview
|EGI HTC Country overview
|-
|-
|/egi/site/
|/egi/site/
|Site
|EGI HTC Site detail
|EGI HTC Site detail
|-
|-
|wlcg/
|wlcg/
|No
|WLCG HTC Overview
|WLCG HTC Overview
|-
|-
|wlcg/country/
|wlcg/country/
|Country
|WLCG HTC Country detail
|WLCG HTC Country detail
|-
|-
|wlcg/site/
|wlcg/site/
|Site
|WLCG HTC Site detail
|WLCG HTC Site detail
|-
|-
|osg/cloud/
|osg/cloud/
|No
|OSG Cloud overview
|OSG Cloud overview
|-
|-
|osg/cloud/site/
|osg/cloud/site/
|Site
|OSG Site detail
|OSG Site detail
|-
|-
|osg/
|osg/
|No
|OSG HTC overview
|OSG HTC overview
|-
|-
|osg/site
|osg/site
|Site
|OSG Site detail
|OSG Site detail
|-
|-
|site_admin/
|site_admin/
|Site
|Site Admin HTC
|Site Admin HTC
|-
|-
|site_admin/cloud/
|site_admin/cloud/
|Site
|Site Admin Cloud
|Site Admin Cloud
|-
|-
|vo_admin/cloud/
|vo_admin/cloud/
|VO
|VO Admin Cloud
|VO Admin Cloud
|-
|-
|vo_admin/cloud/site/
|vo_admin/cloud/site/
|VO, Site
|VO Admin Site detail
|VO Admin Site detail
|-
|-
|vo_admin/cloud/ngi/
|vo_admin/cloud/ngi/
|VO, NGI
|VO Admin NGI detail
|VO Admin NGI detail
|-
|-
|vo_admin/cloud/country/
|vo_admin/cloud/country/
|VO, Country
|VO Admin Country detail
|VO Admin Country detail
|-
|-
|vo_admin/
|vo_admin/
|VO
|VO Admin HTC
|VO Admin HTC
|-
|-
|vo_admin/site/
|vo_admin/site/
|VO, Site
|VO Admin HTC Site detail
|VO Admin HTC Site detail
|-
|-
|vo_admin/ngi/
|vo_admin/ngi/
|VO, NGI
|VO Admin HTC NGI detail
|VO Admin HTC NGI detail
|-
|-
|vo_admin/country/
|vo_admin/country/
|VO, Country
|VO Admin HTC Country detail
|VO Admin HTC Country detail
|-
|-
|user/cloud/
|user/cloud/
|No
|User View Cloud
|User View Cloud
|-
|-
|user/cloud/site/
|user/cloud/site/
|Site
|User View Cloud Site
|User View Cloud Site
|-
|-
|user/cloud/ngi/
|user/cloud/ngi/
|NGI
|User View Cloud NGI
|User View Cloud NGI
|-
|-
|user/cloud/country/
|user/cloud/country/
|Country
|User View Cloud Country
|User View Cloud Country
|-
|-
|user/
|user/
|No
|User View HTC
|User View HTC
|-
|-
|user/site/
|user/site/
|Site
|User View HTC Site
|User View HTC Site
|-
|-
|user/ngi/
|user/ngi/
|NGI
|User View HTC NGI
|User View HTC NGI
|-
|-
|user/country/
|user/country/
|Country
|User View HTC Country
|User View HTC Country
|-
|-
|cloud/
|cloud/
|No
|Cloud NGI Overview
|Cloud NGI Overview
|-
|-
|cloud/countries/
|cloud/countries/
|No
|Cloud Countries Overview
|Cloud Countries Overview
|-
|-
|cloud/site/
|cloud/site/
|Site
|Cloud Site Detail
|Cloud Site Detail
|-
|-
|cloud/ngi/
|cloud/ngi/
|NGI
|Cloud NGI Detail
|Cloud NGI Detail
|-
|-
|cloud/country/
|cloud/country/
|Country
|Cloud Country Detail
|Cloud Country Detail
|-
|-
|wlcg/countries/
|wlcg/countries/
|No
|WLCG Countries Overview
|WLCG Countries Overview
|-
|-
|wlcg/country
|wlcg/country
|Country
|WLCG Country Detail
|WLCG Country Detail
|}
|}

Revision as of 12:01, 8 June 2017

Introduction

The EGI Accounting Portal Data Retrieval API is a REST interface that relies on HTTP commands that share the same channel as the normal user interface. The output can be selectable from CSV (comma separated value) or JSON (JavaScript Object Notation). Both can be readily adapted to whatever internal notation the consumer uses and have support in any relevant language or execution environment.

The basic premise of the API is to append a /csv/ or /json/ to existing URLs from the portal, this makes easy to first manipulate the user interface to get the desired information and then use the URL to get a data endpoint for it. This page will also cover how to change each of the URL components programmatically to achieve the same effect.

The Accounting Portal - technological choices

The Accounting Portal is an AJAX-based we application, but in contrast with many of them offers deep linking by changing the URL dynamically on user selection.

This functionality requires a reasonably recent browser, but since it already requires CORS for basic operation, this functionality is supported by all browsers with CORS already. [1][2]

The non-conforming browsers only represent 5% of the current user base (mostly Opera Mini for mobile phones). Any relevant browser from the last 5 years supports these functionalities.

Basic format of the Accounting Portal URLs

So, for example, to see the Total number of jobs by Operations Centre and Month for EGI Official VOs, we would use a simple URL with the portal WWW domain, a view, and a API-enabling data-selector that enables the API and selects

https://accounting.egi.eu/egi/JSON/

https://<---- Domain --->/<view>/<DS>/

https://accounting.egi.eu/egi/CSV/

Using a URL with only the view and data selector selects default values for each of the selectable parameters in the interface, clicking the “Update” button will update the URL with all selected parameters.

The CPU Efficiency (%) by Resource Centre and Year for EGI Official VOs between June and July 2016 would be: https://accounting.egi.eu/egi/cpueff/SITE/Year/2016/6/2016/7/egi/onlyinfrajobs/CSV/

https://<--- Domain --->/<view>/<query>/<RV>/<CV>/<Date>/<VO>/<Local>/<DS>/

https://accounting.egi.eu/egi/cpueff/SITE/Year/2016/6/2016/7/egi/onlyinfrajobs/JSON/

These URLs were obtained after selecting the relevant options in the web site, and then appended with '/CSV/' or '/JSON/' as required.

Each of the URL segments encodes one aspect, each of which will be covered in a section. '/egi/' --> this is the Path Selector, for some views it would consist on several segments, encondig for example a certain Resource Centre, Country or Discipline. '/cpueff/' ---> would be the selected metric. '/SITE/' is the ROW variable, in this case resource centre. '/Year/' is the COLUMN variable, in this case Year. '/2016/6/' --> is the Year and Month of the start date. '/2016/7/' --> Year and month of the ending date. '/egi/' --> for official VOs. '/onlyinfrajobs/' --> to filter out local jobs. '/CSV/' or '/JSON/' to select the ouput format.

Views

These URL fragments define the selected view and with it the data shown. Most of the views work hierarchically, so from a view of the infrastructure we can navigate to a particular country and inside this a concrete site. The following table shows a list of valid path selectors:

View Data shown
/report/resource_centres/ Infrastructure, type of report Resource centre report
report/disciplines/ Discipline report
wlcg/report/tier2/ WLCG Tier2 Report
wlcg/report/countries/ WLCG Countries Report
wlcg/report/tier1/ WLCG Tier1 Report
report/interngi/ InterNGI Report
report/vomet/ VO activity report
disciplines/cloud/ Cloud disciplines overview
disciplines/cloud/ Cloud Discipline detail
disciplines/ HTC disciplines overview
disciplines/ HTC Discipline detail
/tier1/cloud/ WLCG Cloud Tier1 overview
/tier1/cloud/node/ WLCG Cloud Tier1 node detail
/tier1/cloud/site/ WLCG Cloud Tier1 site detail
/tier1/ WLCG HTC Tier1 overview
/tier1/node/ WLCG HTC Tier1 node detail
/tier1/site/ WLCG HTC Tier1 site detail
/tier2/cloud/ WLCG Cloud Tier2 overview
/tier2/cloud/federation/ WLCG Cloud Tier2 federation detail
/tier2/cloud/country/ WLCG Cloud Tier2 country detail
/tier2/cloud/site/ WLCG Cloud Tier2 site detail
/tier2/ WLCG HTC Tier2 overview
/tier2/federation/ WLCG HTC Tier2 federation detail
/tier2/country/ WLCG HTC Tier2 country detail
/tier2/site/ WLCG HTC Tier2 site detail
/egi/htc/operations_centres/ EGI HTC NGI overview
/egi/ngi/ EGI HTC NGI detail
/egi/country/ EGI HTC Country detail
/egi/countries/ EGI HTC Country overview
/egi/site/ EGI HTC Site detail
wlcg/ WLCG HTC Overview
wlcg/country/ WLCG HTC Country detail
wlcg/site/ WLCG HTC Site detail
osg/cloud/ OSG Cloud overview
osg/cloud/site/ OSG Site detail
osg/ OSG HTC overview
osg/site OSG Site detail
site_admin/ Site Admin HTC
site_admin/cloud/ Site Admin Cloud
vo_admin/cloud/ VO Admin Cloud
vo_admin/cloud/site/ VO Admin Site detail
vo_admin/cloud/ngi/ VO Admin NGI detail
vo_admin/cloud/country/ VO Admin Country detail
vo_admin/ VO Admin HTC
vo_admin/site/ VO Admin HTC Site detail
vo_admin/ngi/ VO Admin HTC NGI detail
vo_admin/country/ VO Admin HTC Country detail
user/cloud/ User View Cloud
user/cloud/site/ User View Cloud Site
user/cloud/ngi/ User View Cloud NGI
user/cloud/country/ User View Cloud Country
user/ User View HTC
user/site/ User View HTC Site
user/ngi/ User View HTC NGI
user/country/ User View HTC Country
cloud/ Cloud NGI Overview
cloud/countries/ Cloud Countries Overview
cloud/site/ Cloud Site Detail
cloud/ngi/ Cloud NGI Detail
cloud/country/ Cloud Country Detail
wlcg/countries/ WLCG Countries Overview
wlcg/country WLCG Country Detail