Difference between revisions of "EGI Quality Criteria Verification"
Line 31: | Line 31: | ||
== Reference Documents == | == Reference Documents == | ||
{| cellspacing="0" cellpadding="3" border="1 | {| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable" | ||
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;" | |- align="left" style="background: none repeat scroll 0% 0% Lightgray;" | ||
! DocDB link | ! DocDB link | ||
Line 52: | Line 52: | ||
== Metrics == | == Metrics == | ||
{| cellspacing="0" cellpadding="3" border="1 | {| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable" | ||
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;" | |- align="left" style="background: none repeat scroll 0% 0% Lightgray;" | ||
! Metric | ! Metric | ||
Line 75: | Line 75: | ||
== Verification engineer skill matrix == | == Verification engineer skill matrix == | ||
{| border="1" style="text-align: center; | {| border="1" class="wikitable" style="text-align: center;" | ||
|+ Verification Cheat Sheet | |+ Verification Cheat Sheet | ||
|- | |- | ||
Line 84: | Line 84: | ||
! width="75" | INFN | ! width="75" | INFN | ||
! width="75" | JÜLICH | ! width="75" | JÜLICH | ||
! width="75" | LIP | ! width="75" | LIP | ||
! width=" | ! width="300" align="left" style="background: none repeat scroll 0% 0% Gray;" | COMMENTS | ||
|- | |- | ||
! align="left" | ARC CE | ! align="left" | ARC CE | ||
Line 93: | Line 93: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" | ARC compute cli | ! align="left" | ARC compute cli | ||
Line 102: | Line 102: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" | ARC Infosys | ! align="left" | ARC Infosys | ||
Line 111: | Line 111: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" | dCache | ! align="left" | dCache | ||
Line 120: | Line 120: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 129: | Line 129: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 138: | Line 138: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 147: | Line 147: | ||
| * | | * | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 156: | Line 156: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 165: | Line 165: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
! align="left" style="background: none repeat scroll 0% 0% Red;" | FTS | ! align="left" style="background: none repeat scroll 0% 0% Red;" | FTS | ||
| | | | ||
| | | | ||
Line 175: | Line 174: | ||
| | | | ||
| | | | ||
| | | | ||
| | |||
|- | |- | ||
! align="left" | DPM | ! align="left" | DPM | ||
Line 183: | Line 183: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 192: | Line 192: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 201: | Line 201: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 210: | Line 210: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" | APEL | ! align="left" | APEL | ||
Line 219: | Line 219: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 228: | Line 228: | ||
| * | | * | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 237: | Line 237: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" | UI | ! align="left" | UI | ||
Line 246: | Line 246: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 255: | Line 255: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" | VOMS_mySQL | ! align="left" | VOMS_mySQL | ||
Line 264: | Line 264: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" style="background: none repeat scroll 0% 0% Red;" | VOMS_Oracle | ! align="left" style="background: none repeat scroll 0% 0% Red;" | VOMS_Oracle | ||
Line 273: | Line 273: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 282: | Line 282: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 291: | Line 291: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 300: | Line 300: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 309: | Line 309: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 318: | Line 318: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 327: | Line 327: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 336: | Line 336: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 345: | Line 345: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 354: | Line 354: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 363: | Line 363: | ||
| | | | ||
| * | | * | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 372: | Line 372: | ||
| | | | ||
| | | | ||
| * | | * | ||
| | | | ||
|- | |- | ||
! align="left" style="background: none repeat scroll 0% 0% Red;" | SAGA-SD | ! align="left" style="background: none repeat scroll 0% 0% Red;" | SAGA-SD | ||
Line 381: | Line 381: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 390: | Line 390: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 399: | Line 399: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 408: | Line 408: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 417: | Line 417: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 426: | Line 426: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|- | |- | ||
Line 435: | Line 435: | ||
| | | | ||
| | | | ||
| | | | ||
| | | | ||
|} | |} | ||
Line 443: | Line 443: | ||
== Verified Products == | == Verified Products == | ||
{| cellspacing="0" cellpadding="3" border="1 | {| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable" | ||
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;" | |- align="left" style="background: none repeat scroll 0% 0% Lightgray;" | ||
! RT link | ! RT link |
Revision as of 16:44, 19 October 2011
Technology | Software Component Delivery | Software Provisioning | UMD Middleware | Cloud Middleware Distribution | Containers Distribution | Technology Glossary |
Quality Assurance | | Quality Criteria Definition | | Quality Criteria Dissemination | | Quality Criteria Verification | | Verifier Guideline | | Verification Testbed | | Glossary |
Objective
The main objective of the TSA2.3 is to verify the quality of the software provided by the TP before entering the SR phase and going to production. By doing so we prevent that software that might work enters into the SR and even goes into production but that doesn´t follow the quality criteria defined in TSA2.2. Some of the reasons for doing the verification before the software enters the stage rollout are:
- Check that the bugs reported in the previous release of the software have been corrected (work in collaboration with DMSU) by the TP.
- Software can work well in the SR but might not have all the functionalities required
- Software might not be safe, well documented, or have the necessary installation rules or licenses
The Verification Process
When a new product is available, the TP has to follow the NSRW. Once the software is correctly uploaded to the repository, the release enters into the verification phase. The requirements are that the TP has to provide all the necessary information to the verifier (QCV) so that the QCV can assess that the TP has tested in advance the quality of the software. Depending on the type of release, different actions will be taken by the QCV, Verification process is described in detail in the EGI Verifier Guideline.
QC Verification Reports
RT workflow and Verification Templates links are available in this page. First of all verifiers must check QC service mapping to know which test must be verified for each product. This service mapping is available here QC Verification service mapping and Verification/Executive Summary templates are available here: QC Verification Templates.
Verifiers must fill all required fields for each product and write a Verification process summary into Executive Summary, this summary should include:
- A short summary of the installation and configuration process: Installation and configuration were successful?, If not explain any issue or bug found. If the product was rejected explain why.
- If its necessary Verifiers should include a comment for StageRollout: Configuration changes or minor issues found verifying the product.
- If a new QC is necessary and is not included, Verifiers must write a comment to SA2.2 to change current QC.
Verification Team
Verifiers team is updated for each UMD release and it is available here: SA2.3 Verifiers List. All verifiers are included in the SSO sw-rel-qc group that includes a mailing list and permissions to modify RT tickets related with the verification process.
Reference Documents
DocDB link | Release Date | Document |
---|---|---|
364 | 11. 08. 2011 | UMD Quality Criteria |
418 | 11. 08. 2011 | UMD Products service mapping |
417 | 11. 08. 2011 | QC Verification Templates |
Metrics
Metric | Description |
---|---|
M.SA2.4 | Number of new releases validated against defined criteria: Measures the workload on the validation team. |
M.SA2.5 | Mean time taken to validate a releas: Indicates how responsive the team is to validating releases. |
M.SA2.6 | Number of releases failing validation: Indicates the quality assurance process of the software providers. |
SLAs
EGI SLAs negotiated with the TPs
Verification engineer skill matrix
Product\Institute | CESGA | IFCA | IFIC | INFN | JÜLICH | LIP | COMMENTS |
---|---|---|---|---|---|---|---|
ARC CE | * | * | |||||
ARC compute cli | * | * | |||||
ARC Infosys | * | * | |||||
dCache | * | ||||||
gLite MPI | * | * | |||||
CREAM_torque | * | * | |||||
CREAM_lsf | * | ||||||
WMS | * | * | |||||
LB | * | * | |||||
FTS | |||||||
DPM | * | ||||||
LFC_mySQL | * | ||||||
LFC_Oracle | |||||||
StoRM | * | * | |||||
APEL | * | ||||||
DGAS | * | ||||||
BDII | * | ||||||
UI | * | ||||||
PX | * | ||||||
VOMS_mySQL | * | ||||||
VOMS_Oracle | |||||||
HYDRA | |||||||
ARGUS | * | ||||||
UNICORE TSI | * | ||||||
UNICORE WS | * | ||||||
UNICORE Client | * | ||||||
UNICORE Registry | * | ||||||
UNICORE Gateway | * | ||||||
UNICORE Hila | * | ||||||
UNICORE xuudb | * | ||||||
UNICORE Uvos | * | ||||||
AMGA | * | ||||||
SAGA-SD | |||||||
GRAM5 | * | ||||||
GridFTP | * | ||||||
myProxy | * | ||||||
RLS | * | ||||||
GridWay | |||||||
SAM | * |
Verified Products
RT link | Verification Date | More information |
---|---|---|
460 | - | CA update, version 1.37-1 |
1125 | - | CA update, version 1.38-1 |
490 | - | SAM Update-6 |
626 | - | SAM Update-7 |
1281 | - | SAM Update-9 |
1729 | 11/04/11 | SAM Update-10 |
1811 | 29/04/11 | SAM Update-10.1 |
1984 | 20/05/11 | SAM Update-11 |
2222 | 01/06/11 | EMI-UI 1.0.0 |