Difference between revisions of "EGI-InSPIRE:SA1 Quality Metrics"
Jump to navigation
Jump to search
Line 1: | Line 1: | ||
Note: P=Public, I=Internal | |||
{| border="1" cellspacing="0" cellpadding="5" align="center" | {| border="1" cellspacing="0" cellpadding="5" align="center" | ||
! Metric ID | ! Metric ID, Description | ||
! | ! Comment | ||
! | ! QR1 | ||
! | ! QR2 | ||
! | ! QR3 | ||
|- | |- | ||
| M.SA1.Usage.1 | | M.SA1.Usage.1, Number of jobs done, per VO and discipline (mean daily/monthly number) - P | ||
| Tool: accounting portal | |||
| | | | ||
| | |||
| | | | ||
|- | |- | ||
| M.SA1.Usage.2 | | M.SA1.Usage.2, Normalised consumed computing capacity - P | ||
| Measured in HEP SPEC. Tool: accounting portal | |||
| | | | ||
| | |||
| | | | ||
|- | |- | ||
| M.SA1.Usage.3 | | M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P | ||
| Requires tool development | |||
| | | | ||
| | |||
| | | | ||
|- | |- | ||
| M.SA1.Size.1 | | M.SA1.Size.1, Total number of production resource centers that are part of the EGI - P | ||
| TSA1.1, Tool: gstat. TSA1.1 QR | |||
| | |||
| TSA1.1 | | | ||
| | |||
|- | |- | ||
| M.SA1.Size.2 | | M.SA1.Size.2, Total number of job slots available in EGI per NGI - P | ||
| TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
| | |||
| TSA1.1 | | | ||
| | |||
|- | |- | ||
| M.SA1.Size.3 | | M.SA1.Size.3, Installed Capacity in HEP SPEC in EGI per NGI - P | ||
| TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
| | |||
| TSA1.1 | | | ||
| | |||
|- | |- | ||
| M.SA1.Size.4 | | M.SA1.Size.4, Installed Disk Capacity (PB) per NGI - P | ||
| TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
| | |||
| TSA1.1 | | | ||
| | |||
|- | |- | ||
| M.SA1.Size.5 | | M.SA1.Size.5, Installed Tape Capacity (PB) per NGI - P | ||
| TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
| | |||
| TSA1.1 | | | ||
| | |||
|- | |- | ||
| M.SA1.OperationalSecurity.1 | | M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I | ||
| TSA1.2, Manual metric (from TSA1.2 QR) | |||
| | |||
| TSA1.2 | | | ||
| | |||
|- | |- | ||
| M.SA1.OperationalSecurity.2 | | M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I | ||
| TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR) | |||
| | |||
| TSA1.2 | | | ||
| | |||
|- | |- | ||
| M.SA1.OperationalSecurity.3 | | M.SA1.OperationalSecurity.3, Number of suspended sites for security issues - I | ||
| TSA1.2, Manual metric (in TSA1.2 QR) | |||
| | |||
| TSA1.2 | | | ||
| | |||
|- | |- | ||
| M.SA1.Integration.1 | | M.SA1.Integration.1, Number of production HPC clusters per NGI - I | ||
| TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR) | |||
| | |||
| TSA1.3 | | | ||
| | |||
|- | |- | ||
| M.SA1.Integration.2 | | M.SA1.Integration.2, Number of production sites supporting MPI - I | ||
| TSA1.3, Passing MPI Nagios tests. Tool: query to the information system Manual metric (NGI QR) | |||
| | |||
| TSA1.3 | | | ||
| | |||
|- | |- | ||
| M.SA1.Integration.3 | | M.SA1.Integration.3, Amount of integrated desktop resources - I | ||
| TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR) | |||
| | |||
| TSA1.3 | | | ||
| | |||
|- | |- | ||
| M.SA1.Integration.4 | | M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users - I | ||
| TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR) | |||
| | |||
| TSA1.3 | | | ||
| | |||
|- | |- | ||
| M.SA1.ServiceValidation.1 | | M.SA1.ServiceValidation.1, Total number of staged rollout components operated per NGI - I | ||
| TSA1.3, Record the number of components participating in each release of a component. Manual metric (TSA1.3 QR) | |||
| | |||
| TSA1.3 | | | ||
| | |||
|- | |- | ||
| M.SA1.ServiceValidation.2 | | M.SA1.ServiceValidation.2, Number of staged rollout releases undertaken & rejected - I | ||
| TSA1.3, Manual metric (TSA1.3 QR) | |||
| | |||
| TSA1.3 | | | ||
| | |||
|- | |- | ||
| M.SA1.Accounting.1 | | M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P | ||
| TSA1.5, Manual metric (TSA1.5 QR) | |||
| | |||
| TSA1.5 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.1 | | M.SA1.Support.1, Number of operational tickets (operational and user) CREATED /SOLVED per Support Unit per month - P | ||
| TSA1.7, Tool: GGUS reporting tool | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.2 | | M.SA1.Support.2, Average/Median Solution Time ticket solution time (business days) per Support Unit per month - P | ||
| TSA1.7, Tool: GGUS reporting tool | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.3 | | M.SA1.Support.3, Number of monthly tickets opened per NGI - P | ||
| TSA1.7, Tool: GGUS reporting tool | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.4 | | M.SA1.Support.4, Assigned ticket Average/Median RESPONSE TIME/SOLUTION TIME per Support Unit per month - P | ||
| TSA1.7, Tool: GGUS reporting tool | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.5 | | M.SA1.Support.5, Number of tickets SOLVED by TPM (1st line support) - I | ||
| TSA1.7, Tool: GGUS reporting tool | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.6 | | M.SA1.Support.6, Ticket assignment time to Support Units by TPM (1st line support) - I | ||
| TSA1.7, Tool: operations dashboard | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.7 | | M.SA1.Support.7,COD Workload per month - I | ||
| TSA1.7, Tool: operations dashboard. TSA1.7 report | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Support.8 | | M.SA1.Support.8, ROD Workload per month per region/NGI - I | ||
| TSA1.7, Tool: operations dashboard. TSA1.7 report | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| MSA1.Support.9 | | MSA1.Support.9, ROD Quality Metrics per month per region/NGI - I | ||
| TSA1.7, Tool: operations dashboard. TSA1.7 report | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Operation.1 | | M.SA1.Operation.1, NGI monthly availability and reliability - P | ||
| TSA1.8, Tool: availability report generator | |||
| | |||
| TSA1.8 | | | ||
| | |||
|- | |- | ||
| M.SA1.Operation.2 | | M.SA1.Operation.2, Number of sites suspended - I | ||
| TSA1.7, Manual metric. TSA1.7 report | |||
| | |||
| TSA1.7 | | | ||
| | |||
|- | |- | ||
| M.SA1.Operation.3 | | M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P | ||
| TSA1.8, Development needed | |||
| | |||
| TSA1.8 | | | ||
| | | | ||
|- | |- | ||
| M.SA1.Operation.4 | | M.SA1.Operation.4, NGI Monthly availability and reliability of core middleware services - P | ||
| TSA1.8, Development needed | |||
| | |||
| TSA1.8 | | | ||
| | | | ||
|- | |- | ||
| M.SA1.Operation.5 | | M.SA1.Operation.5, EGI monthly availability and reliability of core middleware services - P | ||
| TSA1.8, Development needed | |||
| | |||
| TSA1.8 | | | ||
| | | | ||
|- | |- | ||
| M.SA1.Operation.6 | | M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P | ||
| TSA1.8, Development needed | |||
| | |||
| TSA1.8 | | | ||
| | | | ||
|} | |} |
Revision as of 12:12, 10 December 2010
Note: P=Public, I=Internal
Metric ID, Description | Comment | QR1 | QR2 | QR3 |
---|---|---|---|---|
M.SA1.Usage.1, Number of jobs done, per VO and discipline (mean daily/monthly number) - P | Tool: accounting portal | |||
M.SA1.Usage.2, Normalised consumed computing capacity - P | Measured in HEP SPEC. Tool: accounting portal | |||
M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P | Requires tool development | |||
M.SA1.Size.1, Total number of production resource centers that are part of the EGI - P | TSA1.1, Tool: gstat. TSA1.1 QR | |||
M.SA1.Size.2, Total number of job slots available in EGI per NGI - P | TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
M.SA1.Size.3, Installed Capacity in HEP SPEC in EGI per NGI - P | TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
M.SA1.Size.4, Installed Disk Capacity (PB) per NGI - P | TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
M.SA1.Size.5, Installed Tape Capacity (PB) per NGI - P | TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR | |||
M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I | TSA1.2, Manual metric (from TSA1.2 QR) | |||
M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I | TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR) | |||
M.SA1.OperationalSecurity.3, Number of suspended sites for security issues - I | TSA1.2, Manual metric (in TSA1.2 QR) | |||
M.SA1.Integration.1, Number of production HPC clusters per NGI - I | TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR) | |||
M.SA1.Integration.2, Number of production sites supporting MPI - I | TSA1.3, Passing MPI Nagios tests. Tool: query to the information system Manual metric (NGI QR) | |||
M.SA1.Integration.3, Amount of integrated desktop resources - I | TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR) | |||
M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users - I | TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR) | |||
M.SA1.ServiceValidation.1, Total number of staged rollout components operated per NGI - I | TSA1.3, Record the number of components participating in each release of a component. Manual metric (TSA1.3 QR) | |||
M.SA1.ServiceValidation.2, Number of staged rollout releases undertaken & rejected - I | TSA1.3, Manual metric (TSA1.3 QR) | |||
M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P | TSA1.5, Manual metric (TSA1.5 QR) | |||
M.SA1.Support.1, Number of operational tickets (operational and user) CREATED /SOLVED per Support Unit per month - P | TSA1.7, Tool: GGUS reporting tool | |||
M.SA1.Support.2, Average/Median Solution Time ticket solution time (business days) per Support Unit per month - P | TSA1.7, Tool: GGUS reporting tool | |||
M.SA1.Support.3, Number of monthly tickets opened per NGI - P | TSA1.7, Tool: GGUS reporting tool | |||
M.SA1.Support.4, Assigned ticket Average/Median RESPONSE TIME/SOLUTION TIME per Support Unit per month - P | TSA1.7, Tool: GGUS reporting tool | |||
M.SA1.Support.5, Number of tickets SOLVED by TPM (1st line support) - I | TSA1.7, Tool: GGUS reporting tool | |||
M.SA1.Support.6, Ticket assignment time to Support Units by TPM (1st line support) - I | TSA1.7, Tool: operations dashboard | |||
M.SA1.Support.7,COD Workload per month - I | TSA1.7, Tool: operations dashboard. TSA1.7 report | |||
M.SA1.Support.8, ROD Workload per month per region/NGI - I | TSA1.7, Tool: operations dashboard. TSA1.7 report | |||
MSA1.Support.9, ROD Quality Metrics per month per region/NGI - I | TSA1.7, Tool: operations dashboard. TSA1.7 report | |||
M.SA1.Operation.1, NGI monthly availability and reliability - P | TSA1.8, Tool: availability report generator | |||
M.SA1.Operation.2, Number of sites suspended - I | TSA1.7, Manual metric. TSA1.7 report | |||
M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P | TSA1.8, Development needed | |||
M.SA1.Operation.4, NGI Monthly availability and reliability of core middleware services - P | TSA1.8, Development needed | |||
M.SA1.Operation.5, EGI monthly availability and reliability of core middleware services - P | TSA1.8, Development needed | |||
M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P | TSA1.8, Development needed |