Difference between revisions of "EGI-InSPIRE:SA1 Quality Metrics"
Jump to navigation
Jump to search
Line 153: | Line 153: | ||
| | | | ||
|- | |- | ||
| M.SA1.ServiceValidation.1, Total number of staged rollout components operated in EGI - I | | M.SA1.ServiceValidation.1, Total number of patches tested in staged rollout components operated in EGI - I | ||
| TSA1.3, Record the number of components participating in each release of a component. Manual metric (TSA1.3 QR) | | TSA1.3, Record the number of components participating in each release of a component. Manual metric (TSA1.3 QR) | ||
<!-- Value from the task quarterly metric. The total components is the sum of the components declared in the NGI metric --> | <!-- Value from the task quarterly metric. The total components is the sum of the components declared in the NGI metric --> | ||
Line 168: | Line 168: | ||
| 3 | | 3 | ||
| 3 | | 3 | ||
| | |||
| | |||
|- | |||
| M.SA1.ServiceValidation.3, Number of components with EA/Total Number of EAs - I | |||
| TSA1.3, Manual metric (TSA1.3 QR) | |||
<!-- From task quarterly metric --> | |||
| | |||
| | |||
| | |||
| | | | ||
| | | | ||
Line 209: | Line 218: | ||
|- | |- | ||
--> | --> | ||
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support. | | [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.3], Assigned ticket monthly Average RESPONSE TIME (hours) - P | ||
| TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered. | | TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered. | ||
<!-- see Ticket assignment times of TPMs in period in xls file. Median of response time not available --> | <!-- see Ticket assignment times of TPMs in period in xls file. Median of response time not available --> | ||
Line 218: | Line 227: | ||
| | | | ||
|- | |- | ||
| [http://go.egi.eu/tpm M.SA1.Support. | | [http://go.egi.eu/tpm M.SA1.Support.4], Number of tickets SOLVED by TPM (1st line support) per month - I | ||
<!-- the ticket solved by TPM per month can be obtained by the GGUS reporting tool: select "average solution time" and the "TPM" SU, this produces also the number of tickets solved in the month --> | <!-- the ticket solved by TPM per month can be obtained by the GGUS reporting tool: select "average solution time" and the "TPM" SU, this produces also the number of tickets solved in the month --> | ||
| TSA1.7, Tool: GGUS reporting tool | | TSA1.7, Tool: GGUS reporting tool | ||
Line 227: | Line 236: | ||
| | | | ||
|- | |- | ||
| [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support. | | [https://gus.fzk.de/pages/metrics/download_metrics_reports_quarterly.php M.SA1.Support.5], Average-Median ticket assignment time by TPM (1st line support) per month (hours) - I | ||
| TSA1.7, Tool: GGUS montly reports | | TSA1.7, Tool: GGUS montly reports | ||
<!-- see Ticket assignment times of TPMs in period in closing time xls files available at "quarterly metrics reports" --> | <!-- see Ticket assignment times of TPMs in period in closing time xls files available at "quarterly metrics reports" --> | ||
Line 236: | Line 245: | ||
| | | | ||
|- | |- | ||
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support. | | [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.6],COD Workload per month - I | ||
| TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report | | TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report | ||
| 886/188/1742 | | 886/188/1742 | ||
Line 244: | Line 253: | ||
| | | | ||
|- | |- | ||
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support. | | [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.7], EGI ROD Workload per month - I | ||
| TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report | | TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report | ||
| 4535/1532/4277 | | 4535/1532/4277 | ||
Line 252: | Line 261: | ||
| | | | ||
|- | |- | ||
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance MSA1.Support. | | [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance MSA1.Support.8], EGI ROD Quality Metrics per month - I | ||
| TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report | | TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report | ||
| 0.84/0.81/0.89 | | 0.84/0.81/0.89 | ||
Line 269: | Line 278: | ||
|- | |- | ||
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.Operation.2], Number of sites suspended - I | | [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.Operation.2], Number of sites suspended - I | ||
| TSA1.7, | | TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report | ||
| 0/0/0 | | 0/0/0 | ||
| 3/1/2 | | 3/1/2 |
Revision as of 17:10, 23 March 2011
Main | EGI.eu operations services | Support | Documentation | Tools | Activities | Performance | Technology | Catch-all Services | Resource Allocation | Security |
Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3
Metric ID, Description | Comment | QR1 | QR2 | QR3 | QR4 | QR5 |
---|---|---|---|---|---|---|
M.SA1.Usage.1, Average number of jobs "done" per day for all VOs - P | Excluding OPS and DTEAM. Tool: accounting portal | 834,746 | 871,073 | 819,100 | ||
M.SA1.Usage.2, Normalised consumed computing capacity - P | Normalized elapsed time to a reference value of hepspec06 (hours), excluding DTEAM and OPS. Tool: accounting portal | 916,910,352 | 1,415,191,460 | 1,617,306,040 | ||
M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P | Requires tool development | NA | NA | NA | ||
M.SA1.Size.1, Total number of production resource centers that are part of the EGI, Integrated - P | TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count. | 341 | 337 | 340 | ||
M.SA1.Size.2a, Total number of job slots available in EGI - Integrated - P | TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA | 184,844 | 189,596 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588) | 207,203 (non InSPIRE 101,380, tot 308,583) | ||
M.SA1.Size.2b, Total number of job slots available in EGI - Project - P | TSA1.1, Job slots equal logical cpus. Project only includes logical cpus from Resource Infrastructure Providers that are partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. | 277,193 | 296,588 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588) | 308,583 (non InSPIRE 101,380, tot 308,583) | ||
M.SA1.Size.3, Installed Capacity in HEP SPEC in EGI - P | TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. The total HEPSPEC06 are derived by the capacity reported by each NGI (by summing NGI values) | 45,716,437 | ||||
M.SA1.Size.4, Installed Disk Capacity (PB) in EGI - P | TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR | 76.783 | ||||
M.SA1.Size.5, Installed Tape Capacity (PB) in EGI - P | TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR | 59.829 | ||||
M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I | TSA1.2, Manual metric (from TSA1.2 QR) | 0 | 13 | 0 | ||
M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I | TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR) | no SSC | 100% | no SSC | ||
M.SA1.OperationalSecurity.3, Number of suspended sites for security issues - I | TSA1.2, Manual metric (in TSA1.2 QR) | 0 | 0 | 0 | ||
M.SA1.Integration.1, Number of production HPC clusters - I | TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR) | NA | 55 | 54 | ||
M.SA1.Integration.2, Number of production sites supporting MPI - I | TSA1.3, only sites passing MPI Nagios tests are considered. Tool: query to the information system Manual metric (NGI QR) | NA | 73 | 90 | ||
M.SA1.Integration.3, Amount of integrated desktop resources - I | TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR) | NA | 0 | 1562 | ||
M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users (HEPSPEC) - I | TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR) | NA | 246,200 | 16,109 | ||
M.SA1.ServiceValidation.1, Total number of patches tested in staged rollout components operated in EGI - I | TSA1.3, Record the number of components participating in each release of a component. Manual metric (TSA1.3 QR) | 27 (34 comp) | 30 (34 comp) | 15 (19 comp) | ||
M.SA1.ServiceValidation.2, Number of staged rollout releases undertaken & rejected - I | TSA1.3, Manual metric (TSA1.3 QR) | 0 | 3 | 3 | ||
M.SA1.ServiceValidation.3, Number of components with EA/Total Number of EAs - I | TSA1.3, Manual metric (TSA1.3 QR) | |||||
M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P | TSA1.5, Manual metric (TSA1.5 QR) | NA | 62 | 149 (90 RGMA, 62 direct insertion, 56% infrastructure ok) | ||
M.SA1.Support.1, overall average number of GGUS tickets in EGI per month CREATED - P | TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal. | 766/879/1079 | 931/901/954 | 1137/962/1014 | ||
M.SA1.Support.2, Average/Median monthly ticket solution time (hours) - P | TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal (Average h-Median h) | 172-28/108-24/91-39 | missing/97-31/129-39 | 160-32/51-22/45-25 | ||
M.SA1.Support.3, Assigned ticket monthly Average RESPONSE TIME (hours) - P | TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered. | 16/9/15 | missing/10/15 | 15/12/11 | ||
M.SA1.Support.4, Number of tickets SOLVED by TPM (1st line support) per month - I | TSA1.7, Tool: GGUS reporting tool | 1/7/0 | 2/2/2 | 6/9/2 | ||
M.SA1.Support.5, Average-Median ticket assignment time by TPM (1st line support) per month (hours) - I | TSA1.7, Tool: GGUS montly reports | 4-0/2-0/1-0 | missing/1-0/2-0 | 2-0/1-0/2-0 | ||
M.SA1.Support.6,COD Workload per month - I | TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report | 886/188/1742 | 652/591/487 | 764/551/844 | ||
M.SA1.Support.7, EGI ROD Workload per month - I | TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report | 4535/1532/4277 | 2622/2733/1944 | 2943/1912/2090 | ||
MSA1.Support.8, EGI ROD Quality Metrics per month - I | TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report | 0.84/0.81/0.89 | 0.86/0.89/0.90 | 0.90/0.81/0.76 | ||
M.SA1.Operation.1, NGI monthly availability and reliability - P | TSA1.8, Tool: availability report generator. This metric is the arithmetic average of the weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system). | May10/Jun10/Jul10 | Aug10/Sep10/Oct10 | Nov10/Dec10/Jan11 | Feb11 | |
M.SA1.Operation.2, Number of sites suspended - I | TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report | 0/0/0 | 3/1/2 | 1/0/1 | ||
M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P | TSA1.8, Development needed. This metric quantifies the availability and reliability of core tools operated by the NGI such as the NGI monitoring service, and other tools such as the Operations Portal etc. that can be optionally deployed. The EGI summarized value is the arithmetic mean of the NGI availability and reliability scores. | NA | NA | NA | ||
M.SA1.Operation.4, EGI Monthly availability and reliability of core middleware services - P | TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core middleware services operated by EGI such as the catch-all VOMS, the DTEAM VOMS, and other core middleware services needed by EGI. | NA/NA/NA | NA/NA/NA | NA/NA/NA | ||
M.SA1.Operation.5, EGI monthly availability/reliability of site middleware services - P | TSA1.8, monthly report generator. This metric is the arithmetic average of the montly weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system). | 93.298/94.314 | 90.734/91.926 | 92.318/93.327 | ||
M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P | TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core operational tools operated by EGI such as the GOCDB, Operations Portal, GGUS, Accounting central repositories and portal, the central monitoring infrastructure, etc. | NA/NA/NA | NA/NA/NA | NA/NA/NA |