Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI Quality Criteria Verification"

From EGIWiki
Jump to navigation Jump to search
m
Line 39: Line 39:
== Reference Documents  ==
== Reference Documents  ==


{| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable"
{| cellspacing="0" cellpadding="3" border="1" class="wikitable sortable" style="border: 1px solid black;"
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;"
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;"
! DocDB link  
! DocDB link  
Line 60: Line 60:
== Metrics  ==
== Metrics  ==


{| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable"
{| cellspacing="0" cellpadding="3" border="1" class="wikitable sortable" style="border: 1px solid black;"
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;"
|- align="left" style="background: none repeat scroll 0% 0% Lightgray;"
! Metric  
! Metric  
Line 83: Line 83:
== Verification engineer skill matrix  ==
== Verification engineer skill matrix  ==


{| border="1" class="wikitable" style="text-align: center;"
{| border="1" style="text-align: center;" class="wikitable"
|+ Verification Cheat Sheet  
|+ Verification Cheat Sheet  
|-
|-
Line 95: Line 95:
! width="75" | LIP  
! width="75" | LIP  
! width="75" | BADWB  
! width="75" | BADWB  
! width="75" | UTC-N
! width="75" | UTC-N  
! width="300" style="background: none repeat scroll 0% 0% Gray;" | COMMENTS
! width="300" style="background: none repeat scroll 0% 0% Gray;" | COMMENTS
|-
|-
! align="left" | ARC CE  
! align="left" | EMI ARC CE  
|  
|  
| *  
| *  
Line 107: Line 107:
| *  
| *  
|  
|  
|
|  
|
|  
|-
|-
! align="left" | ARC compute cli  
! align="left" | EMI ARC compute cli  
|  
|  
| *  
| *  
Line 118: Line 118:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | ARC Infosys  
! align="left" | EMI ARC Infosys  
|  
|  
| *  
| *  
Line 130: Line 130:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | dCache  
! align="left" | EMI dCache  
| *  
| *  
|  
|  
Line 142: Line 142:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | gLite MPI  
! align="left" | EMI gLite MPI  
| *  
| *  
| *  
| *  
Line 154: Line 154:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | CREAM_torque  
! align="left" | EMI CREAM_torque  
| *  
| *  
| *  
| *  
Line 166: Line 166:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | CREAM_lsf  
! align="left" | EMI CREAM_lsf  
|  
|  
|  
|  
Line 178: Line 178:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | WMS  
! align="left" | EMI WMS  
| *  
| *  
|  
|  
Line 190: Line 190:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | LB  
! align="left" | EMI LB  
| *  
| *  
|  
|  
Line 202: Line 202:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" style="background: none repeat scroll 0% 0% Red;" | FTS  
! align="left" style="background: none repeat scroll 0% 0% Red;" | EMI FTS  
|
|
|  
|  
|  
|  
Line 213: Line 215:
|  
|  
|  
|  
|
|
|  
|  
| No verifiers available yet.
| No verifiers available yet.
|-
|-
! align="left" | DPM  
! align="left" | EMI DPM  
| *  
| *  
|  
|  
Line 226: Line 226:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | LFC_mySQL  
! align="left" | EMI LFC_mySQL  
| *  
| *  
|  
|  
Line 238: Line 238:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | LFC_Oracle  
! align="left" | EMI LFC_Oracle  
|  
|  
|  
|  
Line 250: Line 250:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | StoRM  
! align="left" | EMI StoRM  
|  
|  
| *  
| *  
Line 262: Line 262:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | APEL  
! align="left" | EMI APEL  
| *  
| *  
|  
|  
Line 274: Line 274:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | DGAS  
! align="left" | EMI DGAS  
|  
|  
|  
|  
Line 286: Line 286:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | BDII  
! align="left" | EMI BDII  
|  
|  
|  
|  
Line 298: Line 298:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UI  
! align="left" | EMI UI  
| *  
| *  
|  
|  
Line 310: Line 310:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | PX  
! align="left" | EMI PX  
|  
|  
|  
|  
Line 322: Line 322:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | VOMS_mySQL  
! align="left" | EMI VOMS_mySQL  
|  
|  
|  
|  
Line 334: Line 334:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" style="background: none repeat scroll 0% 0% Red;" | VOMS_Oracle  
! align="left" style="background: none repeat scroll 0% 0% Red;" | EMI VOMS_Oracle  
|
|
|  
|  
|  
|  
Line 345: Line 347:
|  
|  
|  
|  
|
|
|  
|  
| No verifiers available yet. Oracle training required.
| No verifiers available yet. Oracle training required.
|-
|-
! align="left" style="background: none repeat scroll 0% 0% Red;" | HYDRA  
! align="left" style="background: none repeat scroll 0% 0% Red;" | EMI HYDRA  
|
|
|  
|  
|  
|  
Line 357: Line 359:
|  
|  
|  
|  
|
|
|  
|  
| No verifiers available yet.
| No verifiers available yet.
|-
|-
! align="left" | ARGUS  
! align="left" | EMI ARGUS  
| *  
| *  
|  
|  
Line 370: Line 370:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE TSI  
! align="left" | EMI UNICORE TSI  
|  
|  
|  
|  
Line 382: Line 382:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE WS  
! align="left" | EMI UNICORE WS  
|  
|  
|  
|  
Line 394: Line 394:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE Client  
! align="left" | EMI UNICORE Client  
|  
|  
|  
|  
Line 406: Line 406:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE Registry  
! align="left" | EMI UNICORE Registry  
|  
|  
|  
|  
Line 418: Line 418:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE Gateway  
! align="left" | EMI UNICORE Gateway  
|  
|  
|  
|  
Line 430: Line 430:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE Hila  
! align="left" | EMI UNICORE Hila  
|  
|  
|  
|  
Line 442: Line 442:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE xuudb  
! align="left" | EMI UNICORE xuudb  
|  
|  
|  
|  
Line 454: Line 454:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | UNICORE Uvos  
! align="left" | EMI UNICORE Uvos  
|  
|  
|  
|  
Line 466: Line 466:
| *  
| *  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | AMGA  
! align="left" | EMI AMGA  
|  
|  
|  
|  
Line 478: Line 478:
|  
|  
| *  
| *  
|
|  
|
|  
|  
|  
|-
|-
! align="left" style="background: none repeat scroll 0% 0% Red;" | SAGA-SD  
! align="left" style="background: none repeat scroll 0% 0% Red;" | IGE SAGA-SD  
|
|
|  
|  
|  
|  
Line 489: Line 491:
|  
|  
|  
|  
|
|
|  
|  
| No verifiers available yet.
| No verifiers available yet.
|-
|-
! align="left" | GRAM5  
! align="left" | IGE GRAM5  
|  
|  
| *  
| *  
Line 502: Line 502:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | GridFTP  
! align="left" | IGE GridFTP  
| *  
| *  
|  
|  
Line 514: Line 514:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | myProxy  
! align="left" | IGE myProxy  
| *  
| *  
|  
|  
Line 526: Line 526:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | RLS  
! align="left" | IGE RLS  
| *  
| *  
|  
|  
Line 538: Line 538:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | GridWay  
! align="left" | IGE GridWay  
| *  
| *  
|  
|  
Line 550: Line 550:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|-
|-
! align="left" | SAM  
! align="left" | IGE SAM  
| *  
| *  
|  
|  
Line 562: Line 562:
|  
|  
|  
|  
|
|  
|
|  
|  
|  
|}
|}

Revision as of 08:35, 7 June 2012

Technology Software Component Delivery Software Provisioning UMD Middleware Cloud Middleware Distribution Containers Distribution Technology Glossary


Quality Assurance | Quality Criteria Definition | Quality Criteria Dissemination | Quality Criteria Verification | Verifier Guideline | Verification Testbed | Glossary





Objective

The main objective of the TSA2.3 is to verify the quality of the software provided by the TP before entering the SR phase and going to production. By doing so we prevent that software that might work enters into the SR and even goes into production but that doesn´t follow the quality criteria defined in TSA2.2. Some of the reasons for doing the verification before the software enters the stage rollout are:

- Check that the bugs reported in the previous release of the software have been corrected (work in collaboration with DMSU) by the TP.

- Software can work well in the SR but might not have all the functionalities required

- Software might not be safe, well documented, or have the necessary installation rules or licenses

The Verification Process

When a new product is available, the TP has to follow the NSRW. Once the software is correctly uploaded to the repository, the release enters into the verification phase. The requirements are that the TP has to provide all the necessary information to the verifier (QCV) so that the QCV can assess that the TP has tested in advance the quality of the software. Depending on the type of release, different actions will be taken by the QCV, Verification process is described in detail in the EGI Verifier Guideline.

QC Verification Reports

RT workflow and Verification Templates links are available in this page. First of all verifiers must check QC service mapping to know which test must be verified for each product. This service mapping is available here QC Verification service mapping and Verification/Executive Summary templates are available here: QC Verification Templates.

Verifiers must fill all required fields for each product and write a Verification process summary into Executive Summary, this summary should include:

  • A short summary of the installation and configuration process: Installation and configuration were successful?, If not explain any issue or bug found. If the product was rejected explain why.
  • If its necessary Verifiers should include a comment for StageRollout: Configuration changes or minor issues found verifying the product.
  • If a new QC is necessary and is not included, Verifiers must write a comment to SA2.2 to change current QC.


Verification Work Effort Metrics

Each verification process and change is registered by EGI RT. A complete report is generated on a daily basis each midnight. The XLS file can be downloaded from here SA2 Verification Metrics


Verification Team

Verifiers team is updated for each UMD release and it is available here: SA2.3 Verifiers List. All verifiers are included in the SSO sw-rel-qc group that includes a mailing list and permissions to modify RT tickets related with the verification process.

Reference Documents

DocDB link Release Date Document
QCv3 10. 04. 2012 UMD Quality Criteria
418 11. 08. 2011 UMD Products service mapping
417 11. 08. 2011 QC Verification Templates

Metrics

Metric Description
M.SA2.4 Number of new releases validated against defined criteria: Measures the workload on the validation team.
M.SA2.5 Mean time taken to validate a releas: Indicates how responsive the team is to validating releases.
M.SA2.6 Number of releases failing validation: Indicates the quality assurance process of the software providers.

SLAs

EGI SLAs negotiated with the TPs

EMI-EGI SLA

Verification engineer skill matrix

Verification Cheat Sheet
Product\Institute CESGA IFCA IFIC INFN IN2P3 JÜLICH LIP BADWB UTC-N COMMENTS
EMI ARC CE * *
EMI ARC compute cli * *
EMI ARC Infosys * *
EMI dCache *
EMI gLite MPI * *
EMI CREAM_torque * *
EMI CREAM_lsf *
EMI WMS * *
EMI LB * *
EMI FTS No verifiers available yet.
EMI DPM *
EMI LFC_mySQL *
EMI LFC_Oracle *
EMI StoRM * *
EMI APEL *
EMI DGAS *
EMI BDII *
EMI UI *
EMI PX *
EMI VOMS_mySQL *
EMI VOMS_Oracle No verifiers available yet. Oracle training required.
EMI HYDRA No verifiers available yet.
EMI ARGUS *
EMI UNICORE TSI *
EMI UNICORE WS *
EMI UNICORE Client *
EMI UNICORE Registry *
EMI UNICORE Gateway *
EMI UNICORE Hila *
EMI UNICORE xuudb *
EMI UNICORE Uvos *
EMI AMGA *
IGE SAGA-SD No verifiers available yet.
IGE GRAM5 *
IGE GridFTP *
IGE myProxy *
IGE RLS *
IGE GridWay *
IGE SAM *


Early Adopters list

Early Adopters per UMD product