|
|
| Line 1: |
Line 1: |
| {{Template:Tech menubar}} | | {{Tech menubar}} |
| {{TOC_right}} | | {{TOC_right}} |
|
| |
| == Quality Assurance ==
| |
|
| |
| The Quality Assurance unit in EGI defines and executes the processes of
| |
| * structuring, defining and assurance of technical accuracy of Quality Criteria
| |
| * evolving Quality Criteria from a current set to a future set of criteria valid for assurance and control by Technology Providers
| |
|
| |
| === Contact ===
| |
|
| |
| The contact person for Quality Assurance is
| |
| Enol Fernandez del Castillo, enolfc@ifca.SPAMMENOTunican.es
| |
|
| |
| === Structure and Definition of Quality Criteria ===
| |
|
| |
| EGI's Quality Critera are organised in a set of documents. The organisation of Quality Criteria follows the layout of the UMD Roadmap version for which the Quality Criteria [[https://wiki.egi.eu/wiki/EGI_Technology#Evolution_of_Quality_Criteria are valid]].
| |
|
| |
| One document in any given set of Quality Criteria defines ''Generic Quality Criteria'', whereas the remaining documents define the ''Specific Quality Criteria''.
| |
|
| |
| '''Generic Quality Criteria''' apply to all Products delivered to EGI by a Technology Provider. However, some Generic Quality Criteria may not apply to a specific Product because the scope of the Product falls out of the coverage of the pertinent Generic Quality Criteria. The Quality Control team in EGI advises Technology Providers which Generic Quality Criteria apply to Products they wish to make available to EGI.
| |
|
| |
| '''Specific Quality Criteria''' apply to all Products delivered to EGI that implement a specific UMD Capability. Conversely, if a Product implements more than one Capability at once, then ''all'' Specific Quality Criteria of all implemented Capabilities apply to the given Product.
| |
|
| |
| === Evolution of Quality Criteria ===
| |
|
| |
| It is a well-proven fact in Software Engineering that user requirements for any given piece of software change over time. So, as requirements change, so must the software designed to satisfy these requirements. Consequently, Quality Criteria by which the quality of the software is assessed that is verified for production infrastructure availability, must evolve over time, too.
| |
|
| |
| With change of Quality Criteria come multiple sets of Quality Criteria. To deliver software that matches EGI's quality demand, Technology Providers must know well in advance which set of Quality Criteria are in effect at any given point in time.
| |
|
| |
|
| |
| The following rules define the process of Quality Criteria validity:
| |
| * A set of Quality Criteria covers the complete UMD Capabilities.
| |
| * A set of Quality Criteria are valid for 6 calendar months, following the [[https://wiki.egi.eu/wiki/WP5:_Provisioning_the_Software_Infrastructure_(SA2)#Deliverables UMD Roadmap publication schedule]] with an offset of one month.
| |
| * A set of Quality Criteria is stored in EGI's document database as a set of documents grouped in one storage location.
| |
| * A set of Quality Criteria has a shared version, e.g. "QC-1"
| |
| * At most one set of Quality Criteria is valid at any given point in time marked "FINAL" in EGI's document database.
| |
| * There may be more than one set of Quality Criteria in "DRAFT" status, marked as such in the document database.
| |
| * As time passes, FINAL sets of Quality Criteria may become "DEPRECATED" by the next version of Quality Criteria.
| |
| * All DEPRECATED sets of Quality Criteria are kept publicly available for reference.
| |
|
| |
|
| |
| Quality Criteria may evolve to reflect changing user requirements. The process of evolution is captured as follows:
| |
| # Anyone interested in the Quality Criteria definition may review the documents, at any given point in time.
| |
| # Feedback and requests for change must be given through the appropriate channels associated with the reviewer's affiliation
| |
| ## Reviews and comments from the resource operation communities must be given through the OMB
| |
| ## Reviews and comments from the end user communities must be given, via the VRCs, to the UCB
| |
| # UCB and OMB are collating and prioritising the reviews and comments, and discuss them at regular TCB meetings.
| |
| # The TCB decides upon the changes, and in which DRAFT version of the Quality Criteria they shall be reflected.
| |
|
| |
|
| == Quality Control == | | == Quality Control == |
Quality Control
Where Quality Assurance aims to capture and describe the quality that any software delivered to EGI must meet, Quality Control ensures that the software will meet the quality described earlier.
Hence, Quality Control in EGI is involved in two phases of Quality Control in the relationships between EGI and its Technology Providers:
- Proactive collaboration with Technology Providers relating to Test Plan execution and results.
- Independent control of software quality covering key aspects of delivered software.
Controlling software quality is conducted in a two-step process. Both steps complement each other in the aspects of quality they control. Each step is described in greater detail below.
Contact
The contact person for Quality Assurance is
Carlos Fernandez Sanchez, carlosf@cesgaSPAMMENOT.es
Quality Criteria Verification
This first step of active Quality Control within EGI uses an independent reference testbed deployed at IBERGRID premises. Each Product delivered to EGI for quality control will be assessed based on the amount of changes, i.e.e bugs and/or security holes that have been fixed in the update, and the type and amount of new features, if any.
Quality Criteria Verification will focus on functional tests around the areas of change of the Product under verification.
For each Product that is tested in this step:
- Each Verification officer records a detailed report of what has been verified, and the outcome of it.
- An executive report summarises the verification efforts and the outcome (i.e. whether accepted or rejected). Guidance is given
- to StageRollout which aspects of the Product to re-test or pay special attention to,
- to DMSU for newly discovered, not fixed, or non-critical bugs with workarounds available, for future guidance and support to resource sites and users,
- to Quality Assurance for reassessment of existing Quality Criteria, or to develop new Quality Criteria, if applicable.
- to Technology Providers describing in which circumstances which component of the Product has developed the erroneous behaviour.
Each set of verification reports is permanently stored in EGI's document database and made publicly available for review.
Staged-Rollout
Stage-Rollout complements the Quality Criteria Verification step in exposing the Product to real production conditions. Early Adopter sites that are interested in a timely rollout of the Product update take a part of their production infrastructure into a special "sandbox mode" where the Product update is installed under close scrutiny and exposed to a live environment and user requests.
The exact design and process of Staged-Rollout is described in the Operations context since [Staged-Rollout] needs close collaboration and planning within the Operations community.
Similar to Quality Criteria Verification, all Staged-Rollout activity is permanently stored in written reports per Product and will be publicly available for review and reference.
Software Provisioning
2nd line support: Distributed Middleware Support Unit
The wiki is deprecated and due to be decommissioned by the end of September 2022.