PROC05 Software deliverable testing
|EGI-Engage project:||Main page||WP1(NA1)||WP3(JRA1)||WP5(SA1)||PMB||Deliverables and Milestones||Quality Plan||Risk Plan||Data Plan|
|WP2(NA2)||WP4(JRA2)||WP6(SA2)||AMB||Software and services||Metrics||Project Office||Procedures|
|Title||Software deliverable testing|
This procedure is part of Review procedure for deliverables and milestones. It describes how quality check is done for SW deliverables.
Please refer to the EGI Glossary for the definitions of the terms used in this procedure.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", “MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.
Production tools, which are part of EGI Catalogue, should have a production and a testing/devel instance (at least 2 instances in total)
Each product team should choose between two possible ways to verify the quality of its release:
1. Manual test
2. (Semi-)Automatic internal procedure to test the release In both cases, tests and short document must be finished by the deliverable deadline in the DoA.
As for the classical deliverables (the documents), 3 reviewers (1 moderator + 2 reviewers) will be assigned to each software deliverable. The WP manager has the responsibility to identify the reviewers.
In the case of deliverables containing more than 1 software releases, there will be 3 reviewers for deliverable plus supporting testers
||The candidate release should be installed on the testing/devel instance |
The reviewers will perform the validation tests on the candidate release.
Tests will be executed within a week. During this period, the testing/devel instance should not be updated. Reviewers and developers should agree on the week to perform the tests.
Ensure that outcomes of the testing process are part of the short document describing the software release.
(Semi-)Automatic Internal Testing procedure
Propose to its work package leader a (semi-)automatic procedure to verify the quality of its releases. An example of this procedure is a continuous integration system with a set of automatic/manual tests executed against each built. This procedure should be properly documented.
The document should:
• describe the process adopted by the PT to create a new release
• describe the quality tests performed against each release
• contain instructions to roll back to the previous release in case of issues in production and describe how the risk of data loss (e.g. for A/R and accounting) is managed
||In collaboration with the project management, should validate and approve the procedure verifying it can guarantee a good level of quality assessment. |
||Ensure that outcome of this (Semi-)Automatic Internal Testing procedure should be reported in the short document describing the software release (including a reference to the document at point 1)|