Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Accounting Portal API/egi/htc/operations centres/

From EGIWiki
< Accounting Portal API
Revision as of 13:47, 8 June 2017 by Ivandiaz (talk | contribs) (Created page with "= Metric = The Row and Column varaibles define the dimensions in which the data is grouped, for each view usually the data is grouped bidimensionally using a row variable and a c...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Metric

The Row and Column varaibles define the dimensions in which the data is grouped, for each view usually the data is grouped bidimensionally using a row variable and a column variable, the name derived from appearing in the rows and columns of the table, respectively. These are the valid values for both of these:

Metric Name Description njobs Number of jobs Number of discrete jobs launched, independent on completion status. normcpu Normalized Sum CPU Hours of CPU time consumed in calculations by jobs, normalised using the HEPSPEC06 benchmark. sumcpu Sum CPU Hours of not normalised CPU time consumed by jobs. normelap Normalized Sum Elapsed Hours of wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, normalised using the HEPSPEC06 benchmark. normelap_processors Normalized Sum Elapsed * Number of Processors Hours of wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, normalised using the multiplied by the number of logical cores used by each job. elap_processors Sum Elapsed * Number of Processors Hours of not normalised wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, multiplied by the number of logical cores used by the job. sumelap Unnormalized Wall CPU Hours Hours of not normalised wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks. cpueff CPU Efficiency Normalised Sum CPU / (Normalised Sum Elapsed * Number of Processors). It measures the percentage of running time of the process used to do actual computations.

Metric values that are only useable in cloud views: Metric Name Description vm_num Number of Virtual Machines Number of VM instances running in the selected time period. sum_elap Sum Wall hours (Elapsed time) Hours of wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, multiplied by the number of logical cores used by each job.. sum_elap_processors Sum Elapsed * Number of Processors Hours of wall-clock time consumed by jobs, including I/O, OS and other housekeeping tasks, multiplied by the number of logical cores used by each job.. net_in Inbound Network Traffic Inbound network traffic in bytes towards running VM instances. net_out Outbound Network Traffic Outcoming network traffic in bytes from running VM instances. mem Memory Used Amount of memory in  bytes allocated by running VM instances. disk Disk Used Amount of storage in bytes allocated by running VMs instances.


Metric Values that are only useable in restricted views: Metric Name Description mixed Combined Metrics Number of discrete jobs launched, independent on completion status.


Metric Values that are only useable in WLCG views: Metric Name Description njobs Number of jobs Number of discrete jobs launched, independent on completion status. sumcpu Sum CPU Time Hours Hours of CPU Time as reported to APEL by the batch system. sumcpu_days Sum CPU Time Days Days of CPU Time as reported to APEL by the batch system. normcpu Sum CPU Work HS06 Hours (Same as "Normalized Sum CPU"). Sum of CPU Time Hours as reported to APEL by the batch system and multiplied by benchmarked HEPSPEC06 power for a given resource. normcpu_days Sum CPU Work HS06 Days (Same as "Normalized Sum CPU"). Sum of CPU Time Days as reported to APEL by the batch system and multiplied by benchmarked HEPSPEC06. elap_processors Sum Wallclock Time Hours (Same as "Sum elapsed * Number of processors"). Wallclock time as reported by the batch system. For some sites represent raw wallclock time defined as end time stamp of a particular job minus start time stamp and multiplied by number of processors. There are other sites which do not report raw wallclock time, but rather report raw wallclock time scaled by a batch system to a reference node. For such sites this metric currently shows scaled wallclock multiplied by number of processors. elap_processors_days Sum Wallclock Time Days (Same as "Sum elapsed * Number of processors"). Wallclock time as reported by the batch system. For some sites represent raw wallclock time defined as end time stamp of a particular job minus start time stamp and multiplied by number of processors. There are other sites which do not report raw wallclock time, but rather report raw wallclock time scaled by a batch system to a reference node. For such sites this metric currently shows scaled wallclock multiplied by number of processors. normelap_processors Sum Wallclock Work HS06 Hours (Same as "Normalised Sum elapsed * Number of processors"). Wallclock time as reported by the batch system and multiplied by benchmarked HEPSPEC06 power of a given CPU resource and by number of processors. For some sites represent raw wallclock time defined as end time stamp of a particular job minus start time stamp and multiplied by number of processors. There are other sites which do not report raw wallclock time, but rather report raw wallclock time scaled by a batch system to a reference node. For such sites this metric currently shows scaled and Normalised Wallclock multiplied by Number of Processors normelap_processors_days Sum Wallclock Work HS06 Days (Same as "Normalised Sum elapsed * Number of processors"). Wallclock time as reported by the batch system and multiplied by benchmarked HEPSPEC06 power of a given CPU resource and by number of processors. For some sites represent raw wallclock time defined as end time stamp of a particular job minus start time stamp and multiplied by number of processors. There are other sites which do not report raw wallclock time, but rather report raw wallclock time scaled by a batch system to a reference node. For such sites this metric currently shows scaled and Normalised Wallclock multiplied by Number of Processors cpueff CPU Efficiency 100 * Sum CPU Time / Sum (Wallclock Time * Number of Processors). 3.2.1.- Metric Unit Suffix For all metrics with a unit based on hours, there is an optional suffix to indicate a longer time unit, the basic format is <metric>-<unit_suffix> For example: normcpu-month If no suffix is indicated, hours are selected by default:

Suffix Name Description hour Hours Select the default unit of hours, equivalent to not indicating any suffix. day Days Select a time of unit based on days (24 hours) month Months Select a time of unit based on months (720 hours) quarter Quarters Select a time of unit based on quarters (2160 hours) halfyear Half Years Select a time of unit based on half years (4320 hours) year Years Select a time of unit based on years (8640 hours)

Row and Column Variables

The Row and Column varaibles define the dimensions in which the data is grouped, for each view usually the data is grouped bidimensionally using a row variable and a column variable, the name derived from appearing in the rows and columns of the table, respectively. These are the valid values for both of these: Variable Name Description DATE Month Grouping by Month Year Year Grouping by Year 3M Quarter Grouping by Quarter 6M Half-year Grouping by Half-year COUNTRY Country Grouping by Country REGION Operations Centre Grouping by Operations Centre SITE Resource Centre Grouping by Resource Centre VO VO Grouping by VO SubmitHost SubmitHost Grouping by SubmitHost Resource Centre SITE Group by Resource Centre NUMBER PROCESSORS Number of Processors Group by the number of Logical Cores. NODECOUNT Nodes Gorup by Nodes


Variable values that are only useable in cloud views: Variable Name Description STATUS Status Grouping by VM Status vm_id VM Id Grouping by VM ID

Variable Values that are only useable in restricted views: Variable Name Description UserDN User DN Grouping by UserDN UserRole User Role Grouping by User Role

Variable Values that are only useable in WLCG Tier views: Variable Name Description TIER1 TIER1 Grouping by Tier1 Node FEDERATION Federation Grouping by Tier2 Federation

Dates

The format for dates is /StartYear/StartMonth/EndYear/EndMonth/ The date format is: /YYYY/MM/YYYY/MM/ The date indicated by StartYear/StartMonth needs to be BEFORE EndYear/EndMonth.

VO Group Selector

VO Group selects which VOs are selected for the current view, there are five settings: VO Group Selector Meaning lhc The four VOs used on the WLCG Large Hadron Collider (ATLAS, ALICE, LHCb, CMS). top10 Top 10 VOs that in the period and options selected have the highest unnormalised raw CPU value. all All VOs are included. egi VOs officially registered on the EGI Operations Portal. custom Custom set of virtual organisations.


In the case of a “custom” selection, the used format is custom-vo1,vo2,vo3,..,von

This scales to infrastructure views with 200+ VOs.


Local Job Selector

The Local Job selector parameter filters accounting based on its use of local vs infrastructure reosurce

VO Group Selector Meaning onlyinfrajobs Selects jobs that were successfully submitted and completed by a local cluster by submitting them directly to the the batch system of the computing cluster, i.e. bypassing the Compute Element interface. localinfrajobs All jobs executed by the EGI Federation, regardless of the submission mode. onlylocaljobs Selects jobs that were successfully submitted and completed by a local cluster by submitting them directly to the the batch system of the computing cluster, i.e. bypassing the Compute Element interface.


Data Selector

Data Selector Meaning
'/JSON/' Returns data as a text file containing JSON (Java Script Object Notation) data.
'/CSV/' Returns data as a text file containing CSV (Comma Separated Values) data.