Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI-InSPIRE:SA1 Quality Metrics"

From EGIWiki
Jump to navigation Jump to search
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Template:Op menubar}} Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3
{{Template:EGI-Inspire menubar}}


{| cellspacing="0" cellpadding="5" border="1" align="center"
Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3
 
{| class="wikitable"
|-
|-
! style="width: 25%;" | Metric ID, Description  
! style="width: 25%;" | Metric ID, Description  
Line 37: Line 39:
| NA
| NA
|-
|-
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.1], Total number of production resource centers that are part of the EGI, Integrated - P  
| M.SA1.Size.1, Total number of production resource centers that are part of the EGI, Integrated - P  
| TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count. <!-- Metric portal : sum of RCs-->  
| TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count. <!-- Metric portal : sum of RCs-->  
| 341  
| 341  
Line 45: Line 47:
| 346
| 346
|-
|-
| [http://www3.egee.cesga.es/gridsite/metrics/CESGA/egee_view.php M.SA1.Size.2a], Total number of job slots available in EGI - InSPIRE and Integrated  
| M.SA1.Size.2a, Total number of job slots available in EGI - InSPIRE and Integrated  
| Changed from Q9, exluding peer. TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA <!-- Get from GSTAT the sum of all logical, remove the number of logical  CPU from: USA, US, canada, argentina, brazil, chile, china, colombia, mexico, South Africa and Venezuela
| Changed from Q9, exluding peer. TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA <!-- Get from GSTAT the sum of all logical, remove the number of logical  CPU from: USA, US, canada, argentina, brazil, chile, china, colombia, mexico, puerto rico, South Africa, Ukraine, and Venezuela
INCLUDED:australia india  japan korea malaysia New Zeland Philippines thaiwan Thailand China-honk kong Pakistan  
INCLUDED:australia india  japan korea malaysia New Zeland Philippines thaiwan Thailand China-honk kong Pakistan  
israel EGI-InSPIRE partner  
israel EGI-InSPIRE partner  
Line 104: Line 106:
| N/A (ongoing)<br>
| N/A (ongoing)<br>
|-
|-
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.OperationalSecurity.3], Number of suspended sites for security issues - I  
| [[Underperforming_sites_and_suspensions| M.SA1.OperationalSecurity.3]], Number of suspended sites for security issues - I  
| TSA1.2, Manual metric (in TSA1.2 QR)  
| TSA1.2, Manual metric (in TSA1.2 QR)  
| 0  
| 0  
Line 201: Line 203:
| 4-0/3-0/2-0<br>
| 4-0/3-0/2-0<br>
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.6],COD Workload per month - I  
| [[Grid_operations_oversight#ROD_and_COD_Performance| M.SA1.Support.6]],COD Workload per month - I  
| TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report  
| TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report  
| 886/188/1742  
| 886/188/1742  
Line 210: Line 212:
| <br>
| <br>
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance MSA1.Support.8], NGI ROD Quality Index per month - I  
| [[Grid_operations_oversight#ROD_and_COD_Performance| MSA1.Support.8]], NGI ROD Quality Index per month - I  
| TSA1.7 (This metric is collected for the annual report on a table containing the annual values per NGI)
| TSA1.7 (This metric is collected for the annual report on a table containing the annual values per NGI)
|
|
Line 231: Line 233:


|-
|-
| [https://wiki.egi.eu/wiki/Underperforming_sites_and_suspensions M.SA1.Operation.2], Number of sites suspended - I  
| [[Underperforming_sites_and_suspensions| M.SA1.Operation.2]], Number of sites suspended - I  
| TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report  
| TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report  
| 0/0/0  
| 0/0/0  
Line 329: Line 331:


== Dropped metrics ==
== Dropped metrics ==
{| cellspacing="0" cellpadding="5" border="1" align="center"
{| class="wikitable"
|-
|-
! style="width: 25%;" | Metric ID, Description  
! style="width: 25%;" | Metric ID, Description  
Line 371: Line 373:
| 1548<br>
| 1548<br>
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance M.SA1.Support.7], EGI ROD Workload per month - I  
| [[Grid_operations_oversight#ROD_and_COD_Performance| M.SA1.Support.7]], EGI ROD Workload per month - I  
| Dropped before Q9, TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report  
| Dropped before Q9, TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report  
| 4535/1532/4277  
| 4535/1532/4277  
Line 380: Line 382:
| <br>
| <br>
|-
|-
| [https://wiki.egi.eu/wiki/Grid_operations_oversight#ROD_and_COD_Performance MSA1.Support.8], EGI ROD Quality Metrics per month - I  
| [[Grid_operations_oversight#ROD_and_COD_Performance| MSA1.Support.8]], EGI ROD Quality Metrics per month - I  
| Dropped before Q9, TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report  
| Dropped before Q9, TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report  
| 0.84/0.81/0.89  
| 0.84/0.81/0.89  

Latest revision as of 15:16, 6 January 2015

EGI Inspire Main page


Note: P=Public, I=Internal, QR=Quarterly Report, x/y/z=Month1/Month2/Month3

Metric ID, Description Comment QR1 QR2 QR3 QR4 QR5
M.SA1.Usage.1, Average number of jobs "done" per day for all VOs - P Excluding OPS and DTEAM. Tool: accounting portal 834,746 871,073 819,100 960,053 1,029,830
M.SA1.Usage.2, Normalised consumed computing capacity - P Normalized elapsed time to a reference value of hepspec06 (hours), excluding DTEAM and OPS. Tool: accounting portal 916,910,352 1,415,191,460 1,617,306,040 1,965,664,556

2,363,625,828

M.SA1.Usage.3, Normalised Computing power consumed outside of a user’s home country - P Requires tool development NA NA NA NA NA
M.SA1.Size.1, Total number of production resource centers that are part of the EGI, Integrated - P TSA1.1, Tool: gstat. Local Sites that are part of a local grid, not integrated with EGI, do not need to be included in the count. 341 337 340 347 346
M.SA1.Size.2a, Total number of job slots available in EGI - InSPIRE and Integrated Changed from Q9, exluding peer. TSA1.1, Job slots equal logical cpus. Integrated includes logical cpus from peer grids and Resource Infrastructure Providers that are integrated with EGI but not partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. non-InSPIRE means excluding logical CPUs from USA 277,193 (Including peer) 296,588 (Dec 2010: non-InSPIRE: 98611, including peer) 308,583 (non InSPIRE 101,380, including peer) 338,895 (non InSPIRE 99055, including peer) 337,608 (non InSPIRE 89,184, including peer)
M.SA1.Size.2b, Total number of job slots available in EGI - Project - P TSA1.1, Job slots equal logical cpus. Project only includes logical cpus from Resource Infrastructure Providers that are partners of EGI-InSPIRE. Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. 184,844 189,596 (Dec 2010: Inspire: 197977, non-InSPIRE: 98611, TOT: 296588) 217,887 (non InSPIRE 90,696, tot 308,583) 239,840 (non InSPIRE 99,055, tot 338,895) 248,424 (non InSPIRE 89,184)
M.SA1.Size.3, Installed Capacity in HEP-SPEC 06 in EGI - P TSA1.1, Tool: gstat. NGI can amend results reported by tools, and report correct values in its own QR. The total HEPSPEC06 are derived by the capacity reported by each NGI (by summing NGI values)

1,894,634 1,894,861 1,933,746
M.SA1.Size.4, Installed Disk Capacity (PB) in EGI - P TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR

76.783 88.267 106.69
M.SA1.Size.5, Installed Tape Capacity (PB) in EGI - P TSA1.1, Tool: gstat. The overall installed capacity of EGI is derived from the sum of the capacity reported by NGIs. NGI can amend results reported by tools, and report correct values in its own QR

59.829 66.834 67.447
M.SA1.OperationalSecurity.1, Number of Site Security challenge (SSC) made - I TSA1.2, Manual metric (from TSA1.2 QR) 0 13 0 0 40
M.SA1.OperationalSecurity.2, Number of Sites passing one Security Challenge - I TSA1.2, Target: 50% (to be reviewed in the coming months). Manual metric (from TSA1.2 QR) no SSC 100% no SSC no SSC N/A (ongoing)
M.SA1.OperationalSecurity.3, Number of suspended sites for security issues - I TSA1.2, Manual metric (in TSA1.2 QR) 0 0 0 0 0
M.SA1.Integration.1, Number of production HPC clusters - I TSA1.3, HPC clusters is a site with a dedicated high-speed low-latency communications network. Manual metric (NGI QR) NA 55 54 55 56
M.SA1.Integration.2, Number of production sites supporting MPI - I TSA1.3, only sites passing MPI Nagios tests are considered. Tool: query to the information system Manual metric (NGI QR) NA 73 90 96 93
M.SA1.Integration.3, Average amount of resources available through desktop grid - I (# cores / day) TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR)
M.SA1.ServiceValidation.1, Total number of components tested/rejected in staged rollout TSA1.3, Manual metric (TSA1.3 QR) 11/1 10/3 11/2 29/1 54/2
M.SA1.ServiceValidation.2, Number of staged rollout tests undertaken TSA1.3, Manual metric (TSA1.3 QR)- A single patch can be tested by multiple EA sites at a time. This metric counts the number of actual tests performed by the EA sites. 12 14 14 40 81
M.SA1.ServiceValidation.3, Number of  EA teams
TSA1.3, Manual metric (TSA1.3 QR) 26 NA 40 45 46
M.SA1.Support.1, overall average number of GGUS tickets in EGI per month CREATED - P Prior to Q9 was "operational tickets" TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal. 766/879/1079 931/901/954 1137/962/1014 1105/960/891 939/938/903

M.SA1.Support.2, Average/Median monthly ticket solution time (hours) - P TSA1.7, Tool: GGUS mnthly reports. All SUs - including CIC portal (Average h-Median h) 172-28/108-24/91-39 missing/97-31/129-39 160-32/51-22/45-25 393-29/671-25/321/25 178-43/207-46/96-38
M.SA1.Support.3, Assigned ticket monthly Average RESPONSE TIME (hours) - P TSA1.7, Tool: GGUS monthly reports. Only Operations SUs are considered. 16/9/15 missing/10/15 15/12/11 17/18/11 15/19/14
M.SA1.Support.4, Number of tickets SOLVED by TPM (1st line support) per month - I TSA1.7, Tool: GGUS reporting tool 1/7/0 2/2/2 6/9/2 25/19/17 2/11/4
M.SA1.Support.5, Average-Median ticket assignment time by TPM (1st line support) per month (hours) - I TSA1.7, Tool: GGUS montly reports 4-0/2-0/1-0 missing/1-0/2-0 2-0/1-0/2-0 1-0/3-0/3-0 4-0/3-0/2-0
M.SA1.Support.6,COD Workload per month - I TSA1.7, Number of COD issues handled per month. Tool: operations dashboard. TSA1.7 report 886/188/1742 652/591/487 764/551/844 135/363/315 686/791/560
MSA1.Support.8, NGI ROD Quality Index per month - I TSA1.7 (This metric is collected for the annual report on a table containing the annual values per NGI)
M.SA1.Operation.1, NGI monthly availability and reliability - P TSA1.8, Tool: availability report generator. This metric is the arithmetic average of the weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system). May10/Jun10/Jul10 Aug10/Sep10/Oct10 Nov10/Dec10/Jan11 Feb11/Mar11/Apr11

May11/Jun11

/Jul11

M.SA1.Operation.2, Number of sites suspended - I TSA1.7, Only includes sites suspended for operational issues. Manual metric. TSA1.7 report 0/0/0 3/1/2 1/0/1 2/0/0 0/2/7
M.SA1.Operation.3, NGI monthly availability and reliability of core operations tools - P TSA1.8, Development needed. This metric quantifies the availability and reliability of core tools operated by the NGI such as the NGI monitoring service, and other tools such as the Operations Portal etc. that can be optionally deployed. The EGI summarized value is the arithmetic mean of the NGI availability and reliability scores. NA NA NA NA NA
M.SA1.Operation.5, EGI monthly availability/reliability of site middleware services - P TSA1.8, monthly report generator. This metric is the arithmetic average of the montly weigted availability/reliability scored by all EGI certified sites. For each site the weight is the amout of contributed HEP SPE 06 installed capacity (published on the information discovery system). 93.298/94.314 90.734/91.926 92.318/93.327 94.593/95.829 95.290/94.744
M.SA1.Operation.6, EGI monthly availability and reliability of central operations tools - P TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core operational tools operated by EGI such as the GOCDB, Operations Portal, GGUS, Accounting central repositories and portal, the central monitoring infrastructure, etc. NA/NA/NA NA/NA/NA NA/NA/NA NA NA/NA/NA
MSA1.vo.1, Number of VO support units registered in the EGI Helpdesk. Added singe Q9, The metric quantifies the number of VOs that are using the EGI Helpdesk infrastructure. Number of SUs called VOSupport(*)
MSA1.vo.2, Number of tickets assigned to the VO Support support unit, and not re-assigned to another SU. Added singe Q9, The metric quantifies the usage of the EGI Helpdesk to handle the VO's incidents. GGUS report generator.
MSA1.vo.3, New/decommissioned VOs Added singe Q9
MSA1.vo.4, Low/Medium/High Activity VOs Added singe Q9
MSA1.vo.5, international VOs Added singe Q9
MSA1.vo.6, users grouped by discipline Added singe Q9
MSA1.vo.7, number of central VO support services The number of VO services provided centrally by SA1 Added singe Q9
MSA1.vo.8, availability/reliability of central VO support services Added singe Q9

Dropped metrics

Metric ID, Description Comment QR1 QR2 QR3 QR4 QR5
M.SA1.Accounting.1, Number of sites adopting AMQ messaging for Usage Record publication - P Dropped before Q9, TSA1.5, Manual metric (TSA1.5 QR) 62 149 (90 RGMA, 62 direct insertion, 56% infrastructure ok) 241 (29 not yet publishing AMQ, 62 different publishing system with direct insertion)
242 (22 not yet publishing AMQ)
M.SA1.Integration.4, Amount of virtualised installed capacity accessible to EGI users (HEP-SPEC 06) - I Dropped before Q9 TSA1.3, Placeholder: Need to define virtualised resources and how to integrate them. Manual metric (NGI QR) NA 16,000 16,109 16,482 16,695
M.SA1.Operation.4, EGI Monthly availability and reliability of core middleware services - P Dropped before Q9, TSA1.8, Development needed. This metric quantifies the montly availability and reliability (arithmetic mean) of core middleware services operated by EGI such as the catch-all VOMS, the DTEAM VOMS, and other core middleware services needed by EGI. NA/NA/NA NA/NA/NA NA/NA/NA NA NA/NA/NA
M.SA1.Integration.3, Amount of integrated desktop resources - I (HEP_SPEC06) Dropped before Q9, TSA1.3, Placeholder: Need to define desktop resources and how to integrate them. Manual metric (NGI QR) 0 1562 1562 1548
M.SA1.Support.7, EGI ROD Workload per month - I Dropped before Q9, TSA1.7, The overall ROD workload per month adding up the workload of all NGIs. Tool: operations dashboard. TSA1.7 report 4535/1532/4277 2622/2733/1944 2943/1912/2090 1530/1692/2059 2231/1751/1380
MSA1.Support.8, EGI ROD Quality Metrics per month - I Dropped before Q9, TSA1.7, Arithetic mean of all ROD quality metrics. Tool: operations dashboard. TSA1.7 report 0.84/0.81/0.89 0.86/0.89/0.90 0.90/0.81/0.76 0.85/0.82/0,86 0.91/0.94/0.91