EGI Quality Criteria Definition
|Technology||Software Component Delivery||Software Provisioning||UMD Middleware||Cloud Middleware Distribution||Containers Distribution||Technology Glossary|
|Quality Assurance |||Quality Criteria Definition |||Quality Criteria Dissemination |||Quality Criteria Verification |||Verifier Guideline |||Verification Testbed |||Glossary|
The Quality Criteria (QC) Definition Task is responsible of generating the Quality Criteria Documents that drive the Quality Criteria Verification.
Quality Criteria Lifecycle
Quality Criteria definition is a continuous process driven by the requirements of users and operations communities, however the verification of new software releases is done against fixed date releases of the QC documents, due every 6 months. Between major these releases, two draft versions are made available for lightweight review.
The QC releases are done in coordination with the EGI Roadmap and Technology releases.
There are 3 different possible states for the Quality Criteria documents:
- FINAL, meaning that the documents are actively used for verification of software products
- DRAFT, for documents that are in preparation and will be used in the future for verification.
- DEPRECATED, for documents that are no longer used in verification.
Please check the dissemination information in order to ensure that you are using the correct version of the QC documents.
Quality Criteria and UMD Releases
The QC documents collect criteria for capabilities of a specific UMD Major Release. Only one FINAL version of QC documents is active for any given UMD Release. In the case of having more than one major UMD release active, i.e. new products are verified and made available in different UMD repositories, the verification will use the FINAL version of QC documents for each UMD release. Check the dissemination information for determining which QC version should be used for the verification.
Quality Criteria Change Management
Changes in the criteria documents are triggered by one of the following sources of changes:
- Requirements from User Community
- Requirements from Operations Community (especially software incidents found in production)
- Deficiencies in criteria found in Verification or Stage Rollout
- Recommendations and issues found by the Software Vulnerability Group
- Analysis of reference implementations of UMD Capabilities defined in the Roadmap
- Review and analysis of feedback from Technology Providers
Any change to criteria is tracked in the Related Information field that includes any links to direct source of change for the criterion (e.g. RT ticket with user requirement) and the Revision Log field, that records the historic information about the changes in the criterion.
Moreover, the Quality Criteria Release Notes include an overview of the most relevant changes produced in the whole set of criteria documents.
Quality Criteria Metrics
Number of UMD Roadmap Capabilities defined through validation criteria
This metric tracks the coverage of UMD Capabilities with Quality Criteria. The goal is that for each UMD Capability a number of Quality Criteria are recorded that together define the quality of software contributed against that UMD Capability. It is recorded as a percentage of covered capabilities
Number of software incidents found in production that result in changes to quality criteria
This metric tracks the quality of the defined criteria. Any incidents (bugs) found in EGI Infrastructure should result result in a change of the Quality Criteria. The metric considers also security incidents found by the SVG.
Number of "change triggers" that result into an update of the Quality Criteria (Internal)
Similar to the previous one (and XXX it), this internal metric tracks the quality of the defined criteria by counting the number of change triggers (as defined previously in #) produce changes in the Quality Criteria.