EGI Quality Criteria Verification

Technology

Software Component Delivery

Software Provisioning

UMD Middleware

Cloud Middleware Distribution

Containers Distribution

Technology Glossary

Quality Assurance |

Quality Criteria Definition |

Quality Criteria Dissemination |

Quality Criteria Verification |

Verifier Guideline |

Verification Testbed |

Glossary

Objective

The main objective of the TSA2.3 is to verify the quality of the software provided by the TP before entering the SR phase and going to production. By doing so we prevent that software that might work enters into the SR and even goes into production but that doesn´t follow the quality criteria defined in TSA2.2. Some of the reasons for doing the verification before the software enters the stage rollout are:

- Check that the bugs reported in the previous release of the software have been corrected (work in collaboration with DMSU) by the TP.

- Software can work well in the SR but might not have all the functionalities required

- Software might not be safe, well documented, or have the necessary installation rules or licenses

The Verification Process (RT)

When a new product is available, the TP has to follow the NSRW. Once the software is correctly uploaded to the repository, the release enters into the verification phase. The requirements are that the TP has to provide all the necessary information to the verifier (QCV) so that the QCV can assess that the TP has tested in advance the quality of the software. Depending on the type of release, different actions will be taken by the QCV, Verification process has to be as follows:

• Verification team checks that the pre-conditions of the verification phase are met:
  • RT ticket is in state Open.
  • The RolloutProgress is set to Unverified.
  • CommunicationStatus is OK.
  • Owner is set to nobody.

• If these conditions are reached then Verification process could be started.
• Verification team changes RolloutProgress to In Verification to start to work.
• First of all verifier must use an specific template for each product (see 417 and 418).
• Template fields are filled for each test by the verification team:
  • Accepted: Yes, No or does not apply.
  • Tested: By TP or VLD (Validation Team).

• QC Tests are:
  • Mandatory: The release is rejected if fails one or more of these tests.
  • Optional: These tests are not critical and the release could be verified even if it's failing.

• When the report is finished a new Executive Summary is created:
  • Includes a summary of tests failed and passed (Critical and non critical).
  • A list of comments for SR team.

How to publish Verification Reports:

• A new space into DocumentDB is created to store the verification reports, these DocDB fields must be filled:
  • Title: QC Verification Report: <PRODUCT_NAME>-<VERSION>.
  • Verification Executive Summary.
    • First file field is used to upload Executive Summary doc. Executive Summary file names should have this nomenclature: QC_Verification--Executive_Summary_<PRODUCT_NAME>_<VERSION>.doc.
  • Verification Report.
    • Second file filed is used to upload QC Verification report. Verification report file names should have this nomenclature: QC_Verification--Report_<PRUDUCT_NAME>_<VERSION>.doc .

• The documentDB link is used to set QualityCriteriaVerificationReport for each ticket.
  • Keywords field: Space separated keyword names, must include Quality, Criteria, Verification, etc.
  • Field Media type must be set to Document.
  • DocumentDB field status must be set to FINAL.
  • DocumentDB field View must be set to public.
  • Field Modify should be set to inspire-SA2.
  • TOPICS space field must be set to Software Provisioning Verification Reports and WP5.

Finally you can upload your changes to DocumentDB. A new link is created for the new DocDB space.

• The verifier must put this DocumentDB link into QualityCriteriaVerificationReport product RT ticket field.

• If these conditions are reached then Verification Process is done!.
• And the Verification Team changes RolloutProgress to “StageRollout”.

Verification Templates are updated if a new UMD Quality Criteria is released.

Documents

DocDB link	Expected Release Date	Document
240	10. 02. 2011	UMD Quality Criteria
418	10. 03. 2011	EMI service mapping
417	10. 03. 2011	QC Verification Templates

Metrics

Metric	Description
M.SA2.4	Number of new releases validated against defined criteria: Measures the workload on the validation team.
M.SA2.5	Mean time taken to validate a releas: Indicates how responsive the team is to validating releases.
M.SA2.6	Number of releases failing validation: Indicates the quality assurance process of the software providers.

Verified Products

RT link	Verification Date	More information
460	-	CA update, version 1.37-1
1125	-	CA update, version 1.38-1
490	-	SAM Update-6
626	-	SAM Update-7
1281	-	SAM Update-9
1729	11/04/11	SAM Update-10
1811	29/04/11	SAM Update-10.1