EGI Verifier Guideline

From EGIWiki
Jump to: navigation, search
Technology Software Component Delivery Software Provisioning UMD Middleware Cloud Middleware Distribution Containers Distribution Technology Glossary


Software Provisioning menu: Software Provisioning Process UMD Release Process Quality Assurance UMD Staged Rollout



Contents


This guide describes the verification procedure for the latest QC.

Starting verification

Verification of software products is managed through EGI's RT, every product will consist of one or more tickets that include the information about the product and links to the repositories where the packages are available.


External Verification:
If you are performing an external verification, you do not need to manage the RT tickets. Instead you should perform the verification and provide the final report.


In order to start a verification, the verifier must:

  1. Set RT ticket(s) Owner with the current verifier.
  2. Set UMDRelease to the appropriate UMD Release (currently 2 or 3) and save the state.
  3. Change RolloutProgress to In Verification to start the actual verification.

Core Components Verification List

In order to be able to better organize the verification activity a list of core components that MUST to be verified. The list is available here

Installing the products

Each product to be verified must be installed in a controllable environment where tests are easily reproducible. The EGI Verification Testbed may be used for this purpose (check the link for requesting new VMs for the products).

Products must be installed (or upgraded) using the packages from the repository linked in the RT ticket. The URL for this repository can be found at the Custom Fields of the RT ticket within attribute RepositoryURL. The easiest way of adding the repositoy is using the files available at the repofiles directory.

Verification is performed using UMD repositories + OS provided repositories + EPEL (for RH based distributions). No other external repositories may be used.

Once the product is installed (or upgraded), the verifier must perform a basic configuration that allows testing of the product features. This configuration should use the services already existing in the testbed if they are needed. Check EGI Verification Testbed for a list of available services. Support for dteam and ops.vo.ibergrid.eu VOs is expected for all services.

Sample configurations for some of the products are available at configuration-templates github repo.

QC Assessment & Verification report

Each verification must be documented with a final report that includes an executive summary of the result and a summary of the tests performed. The tests to be performed are described in the the current QC document, see also below and at the http://github.com/egi-qc/qc-tests GitHub repository for more information. Verification report template is available at EGI DocDB #1993 in different formats.

The effort dedicated to testing the criteria should be adjusted depending on the type of release:


The effort dedicated to those criteria should be adjusted depending on the type of release:

Generic QC

Tests for the generic QC are described in http://egi-qc.github.io/. Each criterion includes in the What to check and What to report fields describe what needs to be tested and what to include in the verification report respectively. More detailed information about the tests is also available at EGI_QC6_Testing.

Specific QC

Specific QC depends on the product to be verified. There are two criteria in this section:

Handling issues during verification

If the Verifier finds any problems or issues with the product (any of the criteria is not met or problems to install/configure/operate the product), either they are clarified within the ticket by the verification team by using the verifiers list (sw-rel-qc(at)mailman.egi.eu) or, if the problems needs the interaction of the Technology Provider, a GGUS ticket should be opened.

If a GGUS ticket is needed:

Finishing verification

The product is considered as ACCEPTED if all the criteria marked as critical pass. Otherwise it will be REJECTED


Once filled, the verification report must be uploaded as a new doc in DocDB with the following information:

Finally, the result of the verification must be set by changing the RolloutProgress field of the RT ticket(s).

When the process is finished (the product is accepted or rejected) the verifier must also fill the "Time Worked" RT field to account the real hours/minutes spent to finish the verification process.

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Print/export