Difference between revisions of "Accounting Portal API View /egi/cloud/operations centres/"
Line 78: | Line 78: | ||
|Disk Used | |Disk Used | ||
|Amount of storage in bytes allocated by running VMs instances. | |Amount of storage in bytes allocated by running VMs instances. | ||
|- | |||
|'''processors_times_number_of_virtual_machines''' | |||
|Number of Processors * Number of Virtual Machines | |||
|Number of logical cores used by each job multiplied by number of VM instances running in the selected time period. | |||
|} | |} | ||
Line 238: | Line 242: | ||
|Nodes | |Nodes | ||
|Group by Nodes | |Group by Nodes | ||
|- | |||
|'''Flavor''' | |||
|VM Flavor | |||
|Group by VM Flavor determined by number of CPUs and amount of memory | |||
|} | |} | ||
Latest revision as of 12:12, 13 June 2017
This view offers an overview of the EGI Cloud infrastructure viewed by Operations Centres (NGIs).
Metric
The Row and Column variables define the dimensions in which the data are grouped, for each view usually the data are grouped bidimensionally using a row variable and a column variable, the name derived from appearing in the rows and columns of the table, respectively. These are the valid values for both of these:
Metric values that are only useable in cloud views:
Metric | Name | Description |
---|---|---|
number_of_virtual_machines | Number of Virtual Machines | Number of VM instances running in the selected time period. |
sum_elapsed | Sum Wall hours (Elapsed time) | Hours of wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, multiplied by the number of logical cores used by each job.. |
sum_elapsed_times_number_of_processors | Sum Elapsed * Number of Processors | Hours of wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, multiplied by the number of logical cores used by each job.. |
inbound_network_traffic | Inbound Network Traffic | Inbound network traffic in bytes towards running VM instances. |
outbound_network_traffic | Outbound Network Traffic | Outcoming network traffic in bytes from running VM instances. |
memory_used | Memory Used | Amount of memory in bytes allocated by running VM instances. |
disk_used | Disk Used | Amount of storage in bytes allocated by running VMs instances. |
processors_times_number_of_virtual_machines | Number of Processors * Number of Virtual Machines | Number of logical cores used by each job multiplied by number of VM instances running in the selected time period. |
Metric Unit Suffix
For all metrics with a unit based on hours, there is an optional suffix to indicate a longer time unit, the basic format is
<metric>-<unit_suffix>
For example:
normcpu-month
If no suffix is indicated, hours are selected by default:
Suffix | Name | Description |
---|---|---|
hour | Hours | Select the default unit of hours, equivalent to not indicating any suffix. |
day | Days | Select a time of unit based on days (24 hours) |
month | Months | Select a time of unit based on months (720 hours) |
quarter | Quarters | Select a time of unit based on quarters (2160 hours) |
halfyear | Half Years | Select a time of unit based on half years (4320 hours) |
year | Years | Select a time of unit based on years (8640 hours) |
Row and Column Variables
The Row and Column varaibles define the dimensions in which the data is grouped, for each view usually the data is grouped bidimensionally using a row variable and a column variable, the name derived from appearing in the rows and columns of the table, respectively. These are the valid values for both of these:
Variable | Name | Description |
---|---|---|
DATE | Month | Grouping by Month |
Year | Year | Grouping by Year |
3M | Quarter | Grouping by Quarter |
6M | Half-year | Grouping by Half-year |
COUNTRY | Country | Grouping by Country |
REGION | Operations Centre | Grouping by Operations Centre |
SITE | Resource Centre | Grouping by Resource Centre |
VO | VO | Grouping by VO |
SubmitHost | SubmitHost | Grouping by SubmitHost |
Resource Centre | SITE | Group by Resource Centre |
NUMBER PROCESSORS | Number of Processors | Group by the number of Logical Cores. |
NODECOUNT | Nodes | Group by Nodes |
Flavor | VM Flavor | Group by VM Flavor determined by number of CPUs and amount of memory |
Dates
The URL components for dates are:
/StartYear/StartMonth/EndYear/EndMonth/
The date format is:
/YYYY/MM/YYYY/MM/
The date indicated by StartYear/StartMonth needs to be BEFORE EndYear/EndMonth.
VO Group Selector
VO Group selects which VOs are selected for the current view, there are five settings:
VO Group Selector | Meaning |
---|---|
lhc | The four VOs used on the WLCG Large Hadron Collider (ATLAS, ALICE, LHCb, CMS). |
top10 | Top 10 VOs that in the period and options selected have the highest unnormalised raw CPU value. |
all | All VOs are included. |
egi | VOs officially registered on the EGI Operations Portal. |
custom | Custom set of virtual organisations. |
In the case of a “custom” selection, the used format is
custom-vo1,vo2,vo3,..,von
This scales to infrastructure views with 200+ VOs.
Local Job Selector
The Local Job selector parameter filters accounting based on its use of local vs infrastructure reosurce
VO Group Selector | Meaning |
---|---|
onlyinfrajobs | Selects jobs that were successfully submitted and completed by a local cluster by submitting them directly to the the batch system of the computing cluster, i.e. bypassing the Compute Element interface. |
localinfrajobs | All jobs executed by the EGI Federation, regardless of the submission mode. |
onlylocaljobs | Selects jobs that were successfully submitted and completed by a local cluster by submitting them directly to the the batch system of the computing cluster, i.e. bypassing the Compute Element interface. |
Data Selector
Data Selector | Meaning |
---|---|
/JSON/ | Returns data as a text file containing JSON (Java Script Object Notation) data. |
/CSV/ | Returns data as a text file containing CSV (Comma Separated Values) data. |
Output Example
https://accounting-pre.egi.cesga.es/egi/cloud/operations_centres/sum_elapsed/REGION/DATE/2016/6/2017/6/egi/onlyinfrajobs/
[{"2016-06":16566.0805,"2016-07":33131.4865,"2016-08":18803.3853,"2016-09":54002.7052,"2016-10":38730.4261,"2016-11":50568.2954,"2016-12":22064.6672,"2017-01":37691.0306,"2017-02":55277.0269,"2017-
03":22523.3333,"2017-04":21966.9726,"2017-05":25014.1762,"2017-06":672.833,"id":"NGI_CZ","Total":397012,"Percent":37.94}, {"2016-06":9709.2325,"2016-07":1218.8561,"2016-08":1165.82,"2016-09":351.8834,"2016-
10":7051.7999,"2016-11":16162.4554,"2016-12":98.4605,"2017-01":6317.4192,"2017-02":15871.6756,"2017-03":6148.2476,"2017-04":8960.4734,"2017-05":3518.8196,"2017-
06":746.1312,"id":"NGI_DE","Total":77321,"Percent":7.39}, {"2016-06":2840.6012,"2016-07":2167.2436,"2016-08":5610.8828,"2016-09":47610.632,"2016-10":6910.6372,"2016-11":14616.9019,"2016-12":6244.6066,"2017-
01":11622.7904,"2017-02":7160.7765,"2017-03":4907.5302,"2017-04":3182.2267,"2017-05":3091.695,"2017-06":0.7283,"id":"NGI_FRANCE","Total":115967,"Percent":11.08}, {"2016-06":112.9336,"2016-07":510.4345,"2016-
08":121.9189,"2016-09":1278.7322,"2016-10":150.5637,"2016-11":40.475,"2016-12":79.1297,"2017-01":497.7653,"2017-02":2854.5536,"2017-03":2863.9454,"2017-04":971.9317,"2017-05":2681.3315,"2017-
06":64.6917,"id":"NGI_GRNET","Total":12228,"Percent":1.17}, {"2016-06":19777.4007,"2016-07":19158.5089,"2016-08":5434.2817,"2016-09":12933.1304,"2016-10":13174.5589,"2016-11":14816.7715,"2016-12":8753.2237,"2017-
01":15821.0421,"2017-02":32124.624,"2017-03":30268.6339,"2017-04":13574.5244,"2017-05":12759.8678,"2017-06":1247.5154,"id":"NGI_IBERGRID","Total":199844,"Percent":19.1}, {"2016-06":30452.7647,"2016-
07":9720.056,"2016-08":2078.2751,"2016-09":3740.4467,"2016-10":14727.6192,"2016-11":5495.4876,"2016-12":8614.9837,"2017-01":7082.8571,"2017-02":6518.041,"2017-03":29145.0965,"2017-04":1714.3812,"2017-
05":3125.9922,"2017-06":112.2505,"id":"NGI_IT","Total":122528,"Percent":11.71}, {"2016-06":0,"2016-07":0,"2016-08":0,"2016-09":0,"2016-10":0,"2016-11":12.9334,"2016-12":0,"2017-01":0,"2017-02":1994.4864,"2017-
03":0,"2017-04":2131.6364,"2017-05":13424.294,"2017-06":37.9301,"id":"NGI_NL","Total":17601,"Percent":1.68}, {"2016-06":1.0983,"2016-07":1.1108,"2016-08":70.1514,"2016-09":12428.365,"2016-10":4.0342,"2016-
11":30222.1955,"2016-12":9227.0698,"2017-01":5.2008,"2017-02":1777.5755,"2017-03":443.1332,"2017-04":5.1952,"2017-05":6.3216,"2017-06":0.5286,"id":"NGI_PL","Total":54192,"Percent":5.18}, {"2016-06":0,"2016-
07":0,"2016-08":0,"2016-09":0,"2016-10":0,"2016-11":0,"2016-12":0,"2017-01":0,"2017-02":0,"2017-03":2.6689,"2017-04":0.147,"2017-05":347.1633,"2017-06":0.3505,"id":"NGI_RO","Total":350,"Percent":0.03}, {"2016-
06":5.57,"2016-07":153.2714,"2016-08":1.9525,"2016-09":3126.6494,"2016-10":4248.6278,"2016-11":1618.3062,"2016-12":817.6767,"2017-01":1985.3236,"2017-02":13277.3421,"2017-03":2744.0233,"2017-04":1165.2787,"2017-
05":4164.6309,"2017-06":20.6225,"id":"NGI_SK","Total":33329,"Percent":3.18}, {"2016-06":578.9019,"2016-07":1.4286,"2016-08":485.271,"2016-09":988.2849,"2016-10":564.9311,"2016-11":331.1803,"2016-12":2.248,"2017-
01":308.2381,"2017-02":3485.1,"2017-03":337.5105,"2017-04":2.4098,"2017-05":0,"2017-06":0,"id":"NGI_TR","Total":7086,"Percent":0.68}, {"2016-06":2.7111,"2016-07":4.0892,"2016-08":1.5469,"2016-09":65.6381,"2016-
10":82.362,"2016-11":2.4597,"2016-12":4.0206,"2017-01":130.3062,"2017-02":81.7514,"2017-03":24.6775,"2017-04":0,"2017-05":0,"2017-06":0,"id":"NGI_UA","Total":400,"Percent":0.04}, {"2016-06":1.9633,"2016-
07":1.0739,"2016-08":2060.7723,"2016-09":1.0467,"2016-10":1122.0747,"2016-11":2528.3205,"2016-12":2.7743,"2017-01":1.6758,"2017-02":1.2945,"2017-03":799.5251,"2017-04":1061.8458,"2017-05":444.2534,"2017-
06":0.5781,"id":"NGI_UK","Total":8027,"Percent":0.77}, {"2016-06":25.6655,"2016-07":29.7528,"2016-08":28.18,"2016-09":41.4242,"2016-10":38.6489,"2016-11":64.5294,"2016-12":57.9084,"2017-01":61.0263,"2017-
02":51.5576,"2017-03":97.8561,"2017-04":55.6331,"2017-05":51.7389,"2017-06":15.0078,"id":"Russia","Total":619,"Percent":0.06}, {"2016-06":80075,"2016-07":66097,"2016-08":35862,"2016-09":136569,"2016-
10":86806,"2016-11":136480,"2016-12":55967,"2017-01":81525,"2017-02":140476,"2017-03":100306,"2017-04":54793,"2017-05":68630,"2017-06":2919,"id":"Total","Total":1046506,"Percent":""}, {"2016-06":"7.65%","2016-
07":"6.32%","2016-08":"3.43%","2016-09":"13.05%","2016-10":"8.29%","2016-11":"13.04%","2016-12":"5.35%","2017-01":"7.79%","2017-02":"13.42%","2017-03":"9.58%","2017-04":"5.24%","2017-05":"6.56%","2017-
06":"0.28%","id":"Percent","Percent":"","Total":""},
{"id":"xlegend","0":"NGI_CZ","1":"NGI_DE","2":"NGI_FRANCE","3":"NGI_GRNET","4":"NGI_IBERGRID","5":"NGI_IT","6":"NGI_NL","7":"NGI_PL","8":"NGI_RO","9":"NGI_SK","10":"NGI_TR","11":"NGI_UA","12":"NGI_UK","13":"Russia"
}, {"id":"ylegend","0":"2016-06","1":"2016-07","2":"2016-08","3":"2016-09","4":"2016-10","5":"2016-11","6":"2016-12","7":"2017-01","8":"2017-02","9":"2017-03","10":"2017-04","11":"2017-05","12":"2017-
06","13":"id"}, {"id":"var","xrange":"DATE","yrange":"REGION","query":"sum_elap"}]