Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI-InSPIRE:SA1 Quality Metrics"

From EGIWiki
Jump to navigation Jump to search
Line 1: Line 1:
{{Template:Op menubar}}
{{Template:Op menubar}} Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3  
[[Category:metrics]]
Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3


{| border="1" cellspacing="0" cellpadding="5" align="center"
{| cellspacing="0" cellpadding="5" border="1" align="center"
! style="width: 25%" | Metric ID, Description
! style="width: 25%" |Comment
! style="width: 10%" |QR1
! style="width: 10%" |QR2
! style="width: 10%" |QR3
! style="width: 10%" |QR4
! style="width: 10%" |QR5
|-
|-
| M.SA1.Usage.1, Average number of jobs "done" per day for all VOs - P
! style="width: 25%;" | Metric ID, Description
| Excluding OPS and DTEAM. Tool: accounting portal
! style="width: 25%;" | Comment
<!-- accounting portal -->
! style="width: 10%;" | QR1
| 834,746
! style="width: 10%;" | QR2
| 871,073
! style="width: 10%;" | QR3
| 819,100
! style="width: 10%;" | QR4
! style="width: 10%;" | QR5
|-
| M.SA1.Usage.1, Average number of jobs "done" per day for all VOs - P  
| Excluding OPS and DTEAM. Tool: accounting portal <!-- accounting portal -->  
| 834,746  
| 871,073  
| 819,100  
|
|  
|  
|
|-
|-
| M.SA1.Usage.2, Normalised consumed computing capacity - P
| M.SA1.Usage.2, Normalised consumed computing capacity - P  
| Normalized elapsed time to a reference value of hepspec06 (hours), excluding DTEAM and OPS. Tool: accounting portal
| Normalized elapsed time to a reference value of hepspec06 (hours), excluding DTEAM and OPS. Tool: accounting portal  
| 916,910,352
| 916,910,352  
| 1,415,191,460
| 1,415,191,460  
| 1,617,306,040
| 1,617,306,040  
|
|  
|  
|
|-
|-
| M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P
| M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P  
| Requires tool development
| Requires tool development  
| NA
| NA  
| NA
| NA  
| NA
| NA  
|
|  
|  
|
|-
|-  
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.1], Total number of production resource centers that are part of the EGI, Integrated - P  
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.1], Total number of production resource centers that are part of the EGI, Integrated - P  
| TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count.
| TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count. <!-- Metric portal : sum of RCs-->  
<!-- Metric portal : sum of RCs-->
| 341  
| 341
| 337  
| 337
| 340  
| 340
|
|  
|  
|
|-
|-
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.2a], Total number of job slots available in EGI - Integrated - P  
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.2a], Total number of job slots available in EGI - Integrated - P  
| TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA
| TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA <!-- Get from GSTAT the sum of all logical, remove the number of logical  CPU from: USA, US, canada, argentina, brazil, chile, china, colombia, mexico, South Africa
<!-- Get from GSTAT the sum of all logical, remove the number of logical  CPU from: USA, US, canada, argentina, brazil, chile, china, colombia, mexico, South Africa
INCLUDED:
INCLUDED:
australia 220// Asia Pacific ROC,   
australia 220// Asia Pacific ROC,   
Line 62: Line 58:
China-honk kong 2// Asia pacific  
China-honk kong 2// Asia pacific  
Pakistan 524 // Asia pacific
Pakistan 524 // Asia pacific
TOTAL ADDED: 10684-->
TOTAL ADDED: 10684-->  
| 277,193
| 277,193  
| 296,588 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588)
| 296,588 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588)  
| 308,583 (non InSPIRE 101,380, tot 308,583)
| 308,583 (non InSPIRE 101,380, tot 308,583)  
|
|  
|  
|  
|-
|-
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.2b], Total number of job slots available in EGI - Project - P  
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.2b], Total number of job slots available in EGI - Project - P  
| TSA1.1, Job slots equal logical cpus. Project only includes logical cpus from Resource Infrastructure Providers that are partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR.  
| TSA1.1, Job slots equal logical cpus. Project only includes logical cpus from Resource Infrastructure Providers that are partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR.  
| 184,844
| 184,844  
| 189,596 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588)
| 189,596 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588)  
| 217,887 (non InSPIRE 90,696, tot 308,583) <!-- Fixed with AP sites reported above -->
| 217,887 (non InSPIRE 90,696, tot 308,583) <!-- Fixed with AP sites reported above -->  
|
|  
|  
|  
|-
|-
| M.SA1.Size.3, Installed Capacity in HEP SPEC in EGI - P  
| M.SA1.Size.3, Installed Capacity in HEP SPEC in EGI - P  
| TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. The total HEPSPEC06 are derived by the capacity reported by each NGI (by summing NGI values)
| TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. The total HEPSPEC06 are derived by the capacity reported by each NGI (by summing NGI values) <!-- Sum of the values reported by ngi. Where they are not available, use the gstat2 values, selecting the countries -->  
<!-- Sum of the values reported by ngi. Where they are not available, use the gstat2 values, selecting the countries -->
|
|  
|  
| 1,894,634
|  
|  
| 1,894,634
|
|  
|  
|-
|-
| M.SA1.Size.4, Installed Disk Capacity (PB) in EGI - P  
| M.SA1.Size.4, Installed Disk Capacity (PB) in EGI - P  
| TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR
| TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR <!-- Sum of the values reported in NGI quarterly metrics. Where not available (ngi did not filled up the quarter column in time) use the gstat values: online storage -->  
<!-- Sum of the values reported in NGI quarterly metrics. Where not available (ngi did not filled up the quarter column in time) use the gstat values: online storage -->
|  
|  
|  
|  
| 76.783
| 76.783  
|
|  
|  
|  
|-
|-
| M.SA1.Size.5, Installed Tape Capacity (PB) in EGI - P  
| M.SA1.Size.5, Installed Tape Capacity (PB) in EGI - P  
| TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR
| TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR <!-- Sum of the values reported in NGI quarterly metrics. Where not available (ngi did not filled up the quarter column in time) use the gstat values: nearline storage -->  
<!-- Sum of the values reported in NGI quarterly metrics. Where not available (ngi did not filled up the quarter column in time) use the gstat values: nearline storage -->
|  
|  
|  
|  
| 59.829
| 59.829  
|
|  
|  
|  
|-
|-
| M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I
| M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I  
| TSA1.2, Manual metric (from TSA1.2 QR)
| TSA1.2, Manual metric (from TSA1.2 QR) <!-- Get it from the SA1.2 Task report -->  
<!-- Get it from the SA1.2 Task report -->
| 0
| 13
| 0  
| 0  
| 13
| 0
|  
|  
|
|  
|-
|-
| M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I
| M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I  
| TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR)
| TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR)  
| no SSC
| no SSC <!-- Use the value from the SA1.2 Task report-->  
<!-- Use the value from the SA1.2 Task report-->
| 100%  
| 100%
| no SSC  
| no SSC
|
|  
|  
|
|-
|-
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.OperationalSecurity.3], Number of suspended sites for security issues - I  
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.OperationalSecurity.3], Number of suspended sites for security issues - I  
| TSA1.2, Manual metric (in TSA1.2 QR)
| TSA1.2, Manual metric (in TSA1.2 QR)  
| 0
| 0  
| 0
| 0  
| 0
| 0  
|
|  
|  
|
|-
|-
| M.SA1.Integration.1, Number of production HPC clusters - I
| M.SA1.Integration.1, Number of production HPC clusters - I  
| TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR)  
| TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR) <!-- Sum the HPC centers reported in the NGI quarterly metric table, no other sources.-->  
<!-- Sum the HPC centers reported in the NGI quarterly metric table, no other sources.-->
| NA  
| NA  
| 55
| 55  
| 54
| 54  
|
|  
|  
|  
|-
|-
| M.SA1.Integration.2, Number of production sites supporting MPI - I
| M.SA1.Integration.2, Number of production sites supporting MPI - I  
| TSA1.3, only sites passing MPI Nagios tests are considered. Tool: query to the information system Manual metric (NGI QR)
| TSA1.3, only sites passing MPI Nagios tests are considered. Tool: query to the information system Manual metric (NGI QR) <!-- Sum the NGI values reported in the NGI quarterly report. Where not available take the sites tested for MPI (use gridmap)--> <!-- improvement use gocdb or bdii?-->  
<!-- Sum the NGI values reported in the NGI quarterly report. Where not available take the sites tested for MPI (use gridmap)-->
| NA  
<!-- improvement use gocdb or bdii?-->
| 73  
| NA
| 90  
| 73
|  
| 90
|
|  
|  
|-
|-
| M.SA1.Integration.3, Amount of integrated desktop resources - I
| M.SA1.Integration.3, Amount of integrated desktop resources - I  
| TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR)
| TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR) <!-- From NGI quarterly metric. No other sources.-->  
<!-- From NGI quarterly metric. No other sources.-->
| NA  
| NA
| 0  
| 0
| 1562  
| 1562
|  
|  
|  
|  
|-
|-
| M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users (HEPSPEC) - I
| M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users (HEPSPEC) - I  
| TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR)
| TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR) <!-- From NGI quarterly metric . no other sources-->  
<!-- From NGI quarterly metric . no other sources-->
| NA  
| NA
| 246,200  
| 246,200  
| 16,109
| 16,109  
|
|  
|  
|
|-
|-
| M.SA1.ServiceValidation.1, Total number of component patches tested/rejected in staged rollout
| M.SA1.ServiceValidation.1, Total number of component patches tested/rejected in staged rollout  
| TSA1.3, Manual metric (TSA1.3 QR)
| TSA1.3, Manual metric (TSA1.3 QR) <!-- Value from the task quarterly metric. The total components is the sum of the components declared in the NGI metric -->  
<!-- Value from the task quarterly metric. The total components is the sum of the components declared in the NGI metric -->
| 11/1
| 27 (34 comp)
| 10/3
| 30 (34 comp)
| 11/2
| 11 (2 comp)
| 29/1
|  
|  
|
|-
|-
| M.SA1.ServiceValidation.2, Number of staged rollout tests undertaken
| M.SA1.ServiceValidation.2, Number of staged rollout tests undertaken  
| TSA1.3, Manual metric (TSA1.3 QR)- A single patch can be tested by multiple EA sites at a time. This metric counts the number of actual tests performed by the EA sites.
| TSA1.3, Manual metric (TSA1.3 QR)- A single patch can be tested by multiple EA sites at a time. This metric counts the number of actual tests performed by the EA sites. <!-- From task quarterly metric -->  
<!-- From task quarterly metric -->
| 12
| 0
| 3
| 14
| 14
| 14
| 40
|  
|  
|
|-
|-
| M.SA1.ServiceValidation.3, Number of components with EA/Total Number of EAs - I
| M.SA1.ServiceValidation.3, Number of components with EA/Total Number of EAs - I  
| TSA1.3, Manual metric (TSA1.3 QR)
| TSA1.3, Manual metric (TSA1.3 QR) <!-- From task quarterly metric -->  
<!-- From task quarterly metric -->
| NA
|  
| NA
| NA
| 40/45
|  
|  
|
|
|
|-
|-
| M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P  
| M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P  
| TSA1.5, Manual metric (TSA1.5 QR)
| TSA1.5, Manual metric (TSA1.5 QR) <!-- Task quaeterly metric-->  
<!-- Task quaeterly metric-->
| NA  
| NA
| 62  
| 62
| 149 (90 RGMA, 62 direct insertion, 56% infrastructure ok)  
| 149 (90 RGMA, 62 direct insertion, 56% infrastructure ok)
|
|  
|  
|
|-
|-
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.1], overall average number of GGUS tickets in EGI per month CREATED - P  
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.1], overall average number of GGUS tickets in EGI per month CREATED - P <!-- tickets created: this metrics is extracted from the closing time xls files available at "quarterly metrics reports". Alternatively see monthly "metrics report for GGUS" see "Amount of tickets per SU and VO" --> <!-- tickets solved: I can't find a way to extract the number of solved tickets. Unsolved tickets can be extracted from the report generator (count of tickets with status set to 'unsolved') -->  
<!-- tickets created: this metrics is extracted from the closing time xls files available at "quarterly metrics reports". Alternatively see monthly "metrics report for GGUS" see "Amount of tickets per SU and VO" -->
| TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal.  
<!-- tickets solved: I can't find a way to extract the number of solved tickets. Unsolved tickets can be extracted from the report generator (count of tickets with status set to 'unsolved') -->  
| 766/879/1079  
| TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal.
| 931/901/954  
| 766/879/1079
| 1137/962/1014  
| 931/901/954
| 1137/962/1014
|
|  
|  
|-
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.2], Average/Median monthly ticket solution time (hours) - P
<!-- this metrics is extracted from the closing time xls files available at "quarterly metrics reports"-->
| TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal (Average h-Median h)
| 172-28/108-24/91-39
| missing/97-31/129-39
| 160-32/51-22/45-25
|
|  
|  
|-
|-
<!-- the following metrics is redundant (see Support.1)
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.2], Average/Median monthly ticket solution time (hours) - P <!-- this metrics is extracted from the closing time xls files available at "quarterly metrics reports"-->
| M.SA1.Support.3, Number of monthly tickets opened in EGI - P
| TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal (Average h-Median h)
| TSA1.7, Tool: GGUS reporting tool. Only Operations SUs are considered.
| 172-28/108-24/91-39
|
| missing/97-31/129-39
|
| 160-32/51-22/45-25
|  
|  
|
|  
|  
|-
|-
-->
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.3], Assigned ticket monthly Average RESPONSE TIME (hours) - P  
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.3], Assigned ticket monthly Average RESPONSE TIME (hours) - P
| TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered. <!-- see Ticket assignment times of TPMs in period in xls file. Median of response time not available -->  
| TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered.
| 16/9/15  
<!-- see Ticket assignment times of TPMs in period in xls file. Median of response time not available -->
| missing/10/15  
| 16/9/15
| 15/12/11  
| missing/10/15
|
| 15/12/11
|  
|  
|
|-
|-
| [http://go.egi.eu/tpm M.SA1.Support.4], Number of tickets SOLVED by TPM (1st line support) per month - I
| [http://go.egi.eu/tpm M.SA1.Support.4], Number of tickets SOLVED by TPM (1st line support) per month - I <!-- the ticket solved by TPM per month can be obtained by the GGUS reporting tool: select "average solution time" and the "TPM" SU, this produces also the number of tickets solved in the month -->  
<!-- the ticket solved by TPM per month can be obtained by the GGUS reporting tool: select "average solution time" and the "TPM" SU, this produces also the number of tickets solved in the month -->  
| TSA1.7, Tool: GGUS reporting tool  
| TSA1.7, Tool: GGUS reporting tool
| 1/7/0  
| 1/7/0  
| 2/2/2
| 2/2/2  
| 6/9/2
| 6/9/2  
|
|  
|  
|  
|-
|-
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.5], Average-Median ticket assignment time by TPM (1st line support) per month (hours) - I  
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.5], Average-Median ticket assignment time by TPM (1st line support) per month (hours) - I  
| TSA1.7, Tool: GGUS montly reports
| TSA1.7, Tool: GGUS montly reports  
<!-- see Ticket assignment times of TPMs in period in closing time xls files available at "quarterly metrics reports" -->
<!-- see Ticket assignment times of TPMs in period in closing time xls files available at "quarterly metrics reports" -->  
| 4-0/2-0/1-0
 
| missing/1-0/2-0
| 4-0/2-0/1-0  
| 2-0/1-0/2-0
| missing/1-0/2-0  
|
| 2-0/1-0/2-0  
|  
|  
|  
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.6],COD Workload per month - I
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.6],COD Workload per month - I  
| TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report
| TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report  
| 886/188/1742
| 886/188/1742  
| 652/591/487
| 652/591/487  
| 764/551/844
| 764/551/844  
|
|  
|  
|
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.7], EGI ROD Workload per month - I  
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.7], EGI ROD Workload per month - I  
| TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report
| TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report  
| 4535/1532/4277
| 4535/1532/4277  
| 2622/2733/1944
| 2622/2733/1944  
| 2943/1912/2090
| 2943/1912/2090  
|
|  
|  
|
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance MSA1.Support.8], EGI ROD Quality Metrics per month - I  
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance MSA1.Support.8], EGI ROD Quality Metrics per month - I  
| TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report
| TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report  
| 0.84/0.81/0.89
| 0.84/0.81/0.89  
| 0.86/0.89/0.90
| 0.86/0.89/0.90  
| 0.90/0.81/0.76
| 0.90/0.81/0.76  
|
|  
|  
|
|-
|-
| M.SA1.Operation.1, NGI monthly availability and reliability - P
| M.SA1.Operation.1, NGI monthly availability and reliability - P  
| TSA1.8, Tool: availability report generator. This metric is the arithmetic average of the weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system).
| TSA1.8, Tool: availability report generator. This metric is the arithmetic average of the weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system).  
| [https://documents.egi.eu/public/ShowDocument?docid=42 May10]/[https://documents.egi.eu/public/ShowDocument?docid=96 Jun10]/[https://documents.egi.eu/public/ShowDocument?docid=130 Jul10]
| [https://documents.egi.eu/public/ShowDocument?docid=42 May10]/[https://documents.egi.eu/public/ShowDocument?docid=96 Jun10]/[https://documents.egi.eu/public/ShowDocument?docid=130 Jul10]  
| [https://documents.egi.eu/public/ShowDocument?docid=157 Aug10]/[https://documents.egi.eu/public/ShowDocument?docid=219 Sep10]/[https://documents.egi.eu/public/ShowDocument?docid=238 Oct10]
| [https://documents.egi.eu/public/ShowDocument?docid=157 Aug10]/[https://documents.egi.eu/public/ShowDocument?docid=219 Sep10]/[https://documents.egi.eu/public/ShowDocument?docid=238 Oct10]  
| [https://documents.egi.eu/public/ShowDocument?docid=266 Nov10]/[https://documents.egi.eu/public/ShowDocument?docid=299 Dec10]/[https://documents.egi.eu/public/ShowDocument?docid=332 Jan11]
| [https://documents.egi.eu/public/ShowDocument?docid=266 Nov10]/[https://documents.egi.eu/public/ShowDocument?docid=299 Dec10]/[https://documents.egi.eu/public/ShowDocument?docid=332 Jan11]  
| [https://documents.egi.eu/public/ShowDocument?docid=402 Feb11]
| [https://documents.egi.eu/public/ShowDocument?docid=402 Feb11]  
|  
|  
|-
|-
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.Operation.2], Number of sites suspended - I  
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.Operation.2], Number of sites suspended - I  
| TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report
| TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report  
| 0/0/0
| 0/0/0  
| 3/1/2
| 3/1/2  
| 1/0/1
| 1/0/1  
|
|  
|  
|
|-
|-
| M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P  
| M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P  
| TSA1.8, Development needed. This metric quantifies the availability and reliability of core tools operated by the NGI such as the NGI monitoring service, and other tools such as the Operations Portal etc. that can be optionally deployed. The EGI summarized value is the arithmetic mean of the NGI availability and reliability scores.
| TSA1.8, Development needed. This metric quantifies the availability and reliability of core tools operated by the NGI such as the NGI monitoring service, and other tools such as the Operations Portal etc. that can be optionally deployed. The EGI summarized value is the arithmetic mean of the NGI availability and reliability scores.  
| NA
| NA  
| NA
| NA  
| NA
| NA  
|
|  
|  
|
|-
|-
| M.SA1.Operation.4, EGI Monthly availability and reliability of core middleware services - P
| M.SA1.Operation.4, EGI Monthly availability and reliability of core middleware services - P  
| TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core middleware services operated by EGI such as the catch-all VOMS, the DTEAM VOMS, and other core middleware services needed by EGI.
| TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core middleware services operated by EGI such as the catch-all VOMS, the DTEAM VOMS, and other core middleware services needed by EGI.  
| NA/NA/NA
| NA/NA/NA  
| NA/NA/NA
| NA/NA/NA  
| NA/NA/NA
| NA/NA/NA  
|
|  
|  
|  
|-
|-
| [https://documents.egi.eu/public/ShowDocument?docid=415 M.SA1.Operation.5], EGI monthly availability/reliability of site middleware services - P
| [https://documents.egi.eu/public/ShowDocument?docid=415 M.SA1.Operation.5], EGI monthly availability/reliability of site middleware services - P  
| TSA1.8, monthly report generator. This metric is the arithmetic average of the montly weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system).
| TSA1.8, monthly report generator. This metric is the arithmetic average of the montly weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system).  
| 93.298/94.314
| 93.298/94.314  
| 90.734/91.926
| 90.734/91.926  
| 92.318/93.327
| 92.318/93.327  
|
|  
|  
|  
|-
|-
| M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P
| M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P  
| TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core operational tools operated by EGI such as the GOCDB, Operations Portal, GGUS, Accounting central repositories and portal, the central monitoring infrastructure, etc.  
| TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core operational tools operated by EGI such as the GOCDB, Operations Portal, GGUS, Accounting central repositories and portal, the central monitoring infrastructure, etc.  
| NA/NA/NA
| NA/NA/NA  
| NA/NA/NA
| NA/NA/NA  
| NA/NA/NA
| NA/NA/NA  
|
|  
|  
|  
|}
|}
[[Category:Metrics]]

Revision as of 16:54, 26 April 2011

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security 


Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3
Metric ID, Description Comment QR1 QR2 QR3 QR4 QR5
M.SA1.Usage.1, Average number of jobs "done" per day for all VOs - P Excluding OPS and DTEAM. Tool: accounting portal 834,746 871,073 819,100
M.SA1.Usage.2, Normalised consumed computing capacity - P Normalized elapsed time to a reference value of hepspec06 (hours), excluding DTEAM and OPS. Tool: accounting portal 916,910,352 1,415,191,460 1,617,306,040
M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P Requires tool development NA NA NA
M.SA1.Size.1, Total number of production resource centers that are part of the EGI, Integrated - P TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count. 341 337 340
M.SA1.Size.2a, Total number of job slots available in EGI - Integrated - P TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA 277,193 296,588 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588) 308,583 (non InSPIRE 101,380, tot 308,583)
M.SA1.Size.2b, Total number of job slots available in EGI - Project - P TSA1.1, Job slots equal logical cpus. Project only includes logical cpus from Resource Infrastructure Providers that are partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. 184,844 189,596 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588) 217,887 (non InSPIRE 90,696, tot 308,583)
M.SA1.Size.3, Installed Capacity in HEP SPEC in EGI - P TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. The total HEPSPEC06 are derived by the capacity reported by each NGI (by summing NGI values) 1,894,634
M.SA1.Size.4, Installed Disk Capacity (PB) in EGI - P TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR 76.783
M.SA1.Size.5, Installed Tape Capacity (PB) in EGI - P TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR 59.829
M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I TSA1.2, Manual metric (from TSA1.2 QR) 0 13 0
M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR) no SSC 100% no SSC
M.SA1.OperationalSecurity.3, Number of suspended sites for security issues - I TSA1.2, Manual metric (in TSA1.2 QR) 0 0 0
M.SA1.Integration.1, Number of production HPC clusters - I TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR) NA 55 54
M.SA1.Integration.2, Number of production sites supporting MPI - I TSA1.3, only sites passing MPI Nagios tests are considered. Tool: query to the information system Manual metric (NGI QR) NA 73 90
M.SA1.Integration.3, Amount of integrated desktop resources - I TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR) NA 0 1562
M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users (HEPSPEC) - I TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR) NA 246,200 16,109
M.SA1.ServiceValidation.1, Total number of component patches tested/rejected in staged rollout TSA1.3, Manual metric (TSA1.3 QR) 11/1 10/3 11/2 29/1
M.SA1.ServiceValidation.2, Number of staged rollout tests undertaken TSA1.3, Manual metric (TSA1.3 QR)- A single patch can be tested by multiple EA sites at a time. This metric counts the number of actual tests performed by the EA sites. 12 14 14 40
M.SA1.ServiceValidation.3, Number of components with EA/Total Number of EAs - I TSA1.3, Manual metric (TSA1.3 QR) NA NA NA 40/45
M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P TSA1.5, Manual metric (TSA1.5 QR) NA 62 149 (90 RGMA, 62 direct insertion, 56% infrastructure ok)
M.SA1.Support.1, overall average number of GGUS tickets in EGI per month CREATED - P TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal. 766/879/1079 931/901/954 1137/962/1014
M.SA1.Support.2, Average/Median monthly ticket solution time (hours) - P TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal (Average h-Median h) 172-28/108-24/91-39 missing/97-31/129-39 160-32/51-22/45-25
M.SA1.Support.3, Assigned ticket monthly Average RESPONSE TIME (hours) - P TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered. 16/9/15 missing/10/15 15/12/11
M.SA1.Support.4, Number of tickets SOLVED by TPM (1st line support) per month - I TSA1.7, Tool: GGUS reporting tool 1/7/0 2/2/2 6/9/2
M.SA1.Support.5, Average-Median ticket assignment time by TPM (1st line support) per month (hours) - I TSA1.7, Tool: GGUS montly reports 4-0/2-0/1-0 missing/1-0/2-0 2-0/1-0/2-0
M.SA1.Support.6,COD Workload per month - I TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report 886/188/1742 652/591/487 764/551/844
M.SA1.Support.7, EGI ROD Workload per month - I TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report 4535/1532/4277 2622/2733/1944 2943/1912/2090
MSA1.Support.8, EGI ROD Quality Metrics per month - I TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report 0.84/0.81/0.89 0.86/0.89/0.90 0.90/0.81/0.76
M.SA1.Operation.1, NGI monthly availability and reliability - P TSA1.8, Tool: availability report generator. This metric is the arithmetic average of the weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system). May10/Jun10/Jul10 Aug10/Sep10/Oct10 Nov10/Dec10/Jan11 Feb11
M.SA1.Operation.2, Number of sites suspended - I TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report 0/0/0 3/1/2 1/0/1
M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P TSA1.8, Development needed. This metric quantifies the availability and reliability of core tools operated by the NGI such as the NGI monitoring service, and other tools such as the Operations Portal etc. that can be optionally deployed. The EGI summarized value is the arithmetic mean of the NGI availability and reliability scores. NA NA NA
M.SA1.Operation.4, EGI Monthly availability and reliability of core middleware services - P TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core middleware services operated by EGI such as the catch-all VOMS, the DTEAM VOMS, and other core middleware services needed by EGI. NA/NA/NA NA/NA/NA NA/NA/NA
M.SA1.Operation.5, EGI monthly availability/reliability of site middleware services - P TSA1.8, monthly report generator. This metric is the arithmetic average of the montly weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system). 93.298/94.314 90.734/91.926 92.318/93.327
M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core operational tools operated by EGI such as the GOCDB, Operations Portal, GGUS, Accounting central repositories and portal, the central monitoring infrastructure, etc. NA/NA/NA NA/NA/NA NA/NA/NA
Retrieved from "https://wiki.egi.eu/w/index.php?title=EGI-InSPIRE:SA1_Quality_Metrics&oldid=16025"