Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI Quality Criteria Verification"

From EGIWiki
Jump to navigation Jump to search
Line 1: Line 1:
{{Tech menubar}} {{Tech QA submenu}} {{TOC_right}}  
{{Tech menubar}} {{Tech QA submenu}} {{TOC_right}}  


== Objective ==
== Objective ==


The main objective of the TSA2.3 is to verify the quality of the software provided by the TP before entering the SR phase and going to production. By doing so we prevent that software that might work enters into the SR and even goes into production but that doesn´t follow the quality criteria defined in TSA2.2. Some of the reasons for doing the verification before the software enters the stage rollout are:  
The main objective of the TSA2.3 is to verify the quality of the software provided by the TP before entering the SR phase and going to production. By doing so we prevent that software that might work enters into the SR and even goes into production but that doesn´t follow the quality criteria defined in TSA2.2. Some of the reasons for doing the verification before the software enters the stage rollout are:  
Line 11: Line 11:
- Software might not be safe, well documented, or have the necessary installation rules or licenses  
- Software might not be safe, well documented, or have the necessary installation rules or licenses  


== The Verification Process (RT)  ==
== The Verification Process ==


When a new product is available, the TP has to follow the [https://wiki.egi.eu/wiki/NSRW_IMPLEMENTATION_RT NSRW]. Once the software is correctly uploaded to the repository, the release enters into the verification phase. The requirements are that the TP has to provide all the necessary information to the verifier (QCV) so that the QCV can assess that the TP has tested in advance the quality of the software. Depending on the type of release, different actions will be taken by the QCV, Verification process has to be as follows:
When a new product is available, the TP has to follow the [https://wiki.egi.eu/wiki/NSRW_IMPLEMENTATION_RT NSRW]. Once the software is correctly uploaded to the repository, the release enters into the verification phase. The requirements are that the TP has to provide all the necessary information to the verifier (QCV) so that the QCV can assess that the TP has tested in advance the quality of the software. Depending on the type of release, different actions will be taken by the QCV, Verification process is described in detail in the [[EGI Verifier Guideline]].


*Verification team checks that the pre-conditions of the verification phase are met:
== QC Verification Reports  ==
**RT ticket is in state '''Open'''.
**The RolloutProgress is set to '''Unverified'''.
**CommunicationStatus is '''OK'''.
**Owner is set to '''nobody'''.


*If these conditions are reached then Verification process could be started.  
RT workflow and Verification Templates links are available in: [https://wiki.egi.eu/wiki/EGI_Quality_Criteria_Verification Quality Criteria Verification]. First of all verifiers must check QC service mapping to know which test must be verified for each product. This service mapping is available here [https://documents.egi.eu/document/418 QC Verification service mapping] and Verification/Executive Summary templates are available here: [https://documents.egi.eu/document/417 QC Verification Templates].  
*First of all set '''UMDRelease''' to 1 and save the state.  
*Then verification team changes RolloutProgress to '''In Verification''' to start to work.
*Set RT ticket '''Owner''' with the verifier name.  
*First of all verifier must use an specific template for each product (see [https://documents.egi.eu/document/417 Verification Reports and Executive Summary templates] and [https://documents.egi.eu/document/418 EMI QC products mapping]).
*Template fields are filled for each test by the verification team:
**'''Accepted''': Yes, No or does not apply.
**'''Tested''': By TP or VLD (Validation Team).


*QC Tests are:
Verifiers must fill all required fields for each product and write a Verification process summary into Executive Summary, this summary should include:  
**'''Mandatory''': The release is rejected if fails one or more of these tests.
**'''Optional''': These tests are not critical and the release could be verified even if it's failing.


*When the report is finished a new Executive Summary is created:  
*A short summary of the installation and configuration process: Installation and configuration were successful?, If not explain any issue or bug found. If the product was rejected explain why.
**Includes a summary of tests failed and passed (Critical and non critical).  
*If its necessary Verifiers should include a comment for StageRollout: Configuration changes or minor issues found verifying the product.
**A list of comments for SR team.
*If a new QC is necessary and is not included, Verifiers must write a comment to SA2.2 to change current QC.


<br>
== Verification Team ==
Verifiers team is updated for each UMD release and it is available here: [https://documents.egi.eu/document/514 SA2.3 Verifiers List]


=== How to publish Verification Reports:  ===
== Reference Documents  ==
 
*A new space into [https://documents.egi.eu/secure/DocumentAddForm?mode=add DocumentDB] is created to store the verification reports, these DocDB fields must be filled:
**'''Title''': ''QC Verification Report: &lt;PRODUCT_NAME&gt;-&lt;VERSION&gt;''.
**Verification Executive Summary.
***First file field is used to upload Executive Summary doc. Executive Summary file names should have this nomenclature: ''QC_Verification--Executive_Summary_&lt;PRODUCT_NAME&gt;_&lt;VERSION&gt;.doc''.
**Verification Report.
***Second file filed is used to upload QC Verification report. Verification report file names should have this nomenclature: ''QC_Verification--Report_&lt;PRUDUCT_NAME&gt;_&lt;VERSION&gt;.doc ''.
 
<br>
 
*The documentDB link is used to set QualityCriteriaVerificationReport for each ticket.
**'''Keywords''' field: Space separated keyword names, must include ''Quality'', ''Criteria'', ''Verification'', etc.
**Field '''Media type''' must be set to '''Document'''.
**DocumentDB field '''status''' must be set to '''FINAL'''.
**DocumentDB field '''View''' must be set to '''public'''.
**Field '''Modify''' should be set to '''inspire-SA2'''.
**TOPICS space field must be set to ''Software Provisioning Verification Reports'' and ''WP5''.
 
Finally you can upload your changes to DocumentDB. A new link is created for the new DocDB space.<br>
 
*The verifier must put this DocumentDB link into '''QualityCriteriaVerificationReport''' product RT ticket field.
 
<br>
 
*If these conditions are reached then Verification Process is done!.
*And the Verification Team changes RolloutProgress to “StageRollout”.
 
[[Verification Templates are updated if a new UMD Quality Criteria is released. (textiles)|Verification Templates are updated if a new UMD Quality Criteria is released.]]
 
== Documents  ==


{| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable"
{| cellspacing="0" cellpadding="3" border="1" style="border: 1px solid black;" class="wikitable sortable"
Line 90: Line 48:
| QC Verification Templates
| QC Verification Templates
|}
|}
<br>


== Metrics  ==
== Metrics  ==
Line 109: Line 65:
| Number of releases failing validation: Indicates the quality assurance process of the software providers.
| Number of releases failing validation: Indicates the quality assurance process of the software providers.
|}
|}
<br>


== Verified Products  ==
== Verified Products  ==

Revision as of 18:30, 2 June 2011

Technology Software Component Delivery Software Provisioning UMD Middleware Cloud Middleware Distribution Containers Distribution Technology Glossary


Quality Assurance | Quality Criteria Definition | Quality Criteria Dissemination | Quality Criteria Verification | Verifier Guideline | Verification Testbed | Glossary





Objective

The main objective of the TSA2.3 is to verify the quality of the software provided by the TP before entering the SR phase and going to production. By doing so we prevent that software that might work enters into the SR and even goes into production but that doesn´t follow the quality criteria defined in TSA2.2. Some of the reasons for doing the verification before the software enters the stage rollout are:

- Check that the bugs reported in the previous release of the software have been corrected (work in collaboration with DMSU) by the TP.

- Software can work well in the SR but might not have all the functionalities required

- Software might not be safe, well documented, or have the necessary installation rules or licenses

The Verification Process

When a new product is available, the TP has to follow the NSRW. Once the software is correctly uploaded to the repository, the release enters into the verification phase. The requirements are that the TP has to provide all the necessary information to the verifier (QCV) so that the QCV can assess that the TP has tested in advance the quality of the software. Depending on the type of release, different actions will be taken by the QCV, Verification process is described in detail in the EGI Verifier Guideline.

QC Verification Reports

RT workflow and Verification Templates links are available in: Quality Criteria Verification. First of all verifiers must check QC service mapping to know which test must be verified for each product. This service mapping is available here QC Verification service mapping and Verification/Executive Summary templates are available here: QC Verification Templates.

Verifiers must fill all required fields for each product and write a Verification process summary into Executive Summary, this summary should include:

  • A short summary of the installation and configuration process: Installation and configuration were successful?, If not explain any issue or bug found. If the product was rejected explain why.
  • If its necessary Verifiers should include a comment for StageRollout: Configuration changes or minor issues found verifying the product.
  • If a new QC is necessary and is not included, Verifiers must write a comment to SA2.2 to change current QC.

Verification Team

Verifiers team is updated for each UMD release and it is available here: SA2.3 Verifiers List

Reference Documents

DocDB link Expected Release Date Document
240 10. 02. 2011 UMD Quality Criteria
418 10. 03. 2011 EMI service mapping
417 10. 03. 2011 QC Verification Templates

Metrics

Metric Description
M.SA2.4 Number of new releases validated against defined criteria: Measures the workload on the validation team.
M.SA2.5 Mean time taken to validate a releas: Indicates how responsive the team is to validating releases.
M.SA2.6 Number of releases failing validation: Indicates the quality assurance process of the software providers.

Verified Products

RT link Verification Date More information
460 - CA update, version 1.37-1
1125 - CA update, version 1.38-1
490 - SAM Update-6
626 - SAM Update-7
1281 - SAM Update-9
1729 11/04/11 SAM Update-10
1811 29/04/11 SAM Update-10.1
1984 20/05/11 SAM Update-11
2222 01/06/11 EMI-UI 1.0.0
Retrieved from "https://wiki.egi.eu/w/index.php?title=EGI_Quality_Criteria_Verification&oldid=18611"