Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI Software Provisioning

From EGIWiki
Jump to navigation Jump to search


Quality Assurance

The Quality Assurance unit in EGI defines and executes the processes of

  • structuring, defining and assurance of technical accuracy of Quality Criteria
  • evolving Quality Criteria from a current set to a future set of criteria valid for assurance and control by Technology Providers

Contact

The contact person for Quality Assurance is Enol Fernandez del Castillo, enolfc@ifca.SPAMMENOTunican.es

Structure and Definition of Quality Criteria

EGI's Quality Critera are organised in a set of documents. The organisation of Quality Criteria follows the layout of the UMD Roadmap version for which the Quality Criteria [are valid].

One document in any given set of Quality Criteria defines Generic Quality Criteria, whereas the remaining documents define the Specific Quality Criteria.

Generic Quality Criteria apply to all Products delivered to EGI by a Technology Provider. However, some Generic Quality Criteria may not apply to a specific Product because the scope of the Product falls out of the coverage of the pertinent Generic Quality Criteria. The Quality Control team in EGI advises Technology Providers which Generic Quality Criteria apply to Products they wish to make available to EGI.

Specific Quality Criteria apply to all Products delivered to EGI that implement a specific UMD Capability. Conversely, if a Product implements more than one Capability at once, then all Specific Quality Criteria of all implemented Capabilities apply to the given Product.

Evolution of Quality Criteria

It is a well-proven fact in Software Engineering that user requirements for any given piece of software change over time. So, as requirements change, so must the software designed to satisfy these requirements. Consequently, Quality Criteria by which the quality of the software is assessed that is verified for production infrastructure availability, must evolve over time, too.

With change of Quality Criteria come multiple sets of Quality Criteria. To deliver software that matches EGI's quality demand, Technology Providers must know well in advance which set of Quality Criteria are in effect at any given point in time.


The following rules define the process of Quality Criteria validity:

  • A set of Quality Criteria covers the complete UMD Capabilities.
  • A set of Quality Criteria are valid for 6 calendar months, following the [UMD Roadmap publication schedule] with an offset of one month.
  • A set of Quality Criteria is stored in EGI's document database as a set of documents grouped in one storage location.
  • A set of Quality Criteria has a shared version, e.g. "QC-1"
  • At most one set of Quality Criteria is valid at any given point in time marked "FINAL" in EGI's document database.
  • There may be more than one set of Quality Criteria in "DRAFT" status, marked as such in the document database.
  • As time passes, FINAL sets of Quality Criteria may become "DEPRECATED" by the next version of Quality Criteria.
  • All DEPRECATED sets of Quality Criteria are kept publicly available for reference.


Quality Criteria may evolve to reflect changing user requirements. The process of evolution is captured as follows:

  1. Anyone interested in the Quality Criteria definition may review the documents, at any given point in time.
  2. Feedback and requests for change must be given through the appropriate channels associated with the reviewer's affiliation
    1. Reviews and comments from the resource operation communities must be given through the OMB
    2. Reviews and comments from the end user communities must be given, via the VRCs, to the UCB
  3. UCB and OMB are collating and prioritising the reviews and comments, and discuss them at regular TCB meetings.
  4. The TCB decides upon the changes, and in which DRAFT version of the Quality Criteria they shall be reflected.

Quality Control

Where Quality Assurance aims to capture and describe the quality that any software delivered to EGI must meet, Quality Control ensures that the software will meet the quality described earlier.

Hence, Quality Control in EGI is involved in two phases of Quality Control in the relationships between EGI and its Technology Providers:

  • Proactive collaboration with Technology Providers relating to Test Plan execution and results.
  • Independent control of software quality covering key aspects of delivered software.

Controlling software quality is conducted in a two-step process. Both steps complement each other in the aspects of quality they control. Each step is described in greater detail below.

Contact

The contact person for Quality Assurance is Carlos Fernandez Sanchez, carlosf@cesgaSPAMMENOT.es

Quality Criteria Verification

This first step of active Quality Control within EGI uses an independent reference testbed deployed at IBERGRID premises. Each Product delivered to EGI for quality control will be assessed based on the amount of changes, i.e.e bugs and/or security holes that have been fixed in the update, and the type and amount of new features, if any.

Quality Criteria Verification will focus on functional tests around the areas of change of the Product under verification.

For each Product that is tested in this step:

  • Each Verification officer records a detailed report of what has been verified, and the outcome of it.
  • An executive report summarises the verification efforts and the outcome (i.e. whether accepted or rejected). Guidance is given
    • to StageRollout which aspects of the Product to re-test or pay special attention to,
    • to DMSU for newly discovered, not fixed, or non-critical bugs with workarounds available, for future guidance and support to resource sites and users,
    • to Quality Assurance for reassessment of existing Quality Criteria, or to develop new Quality Criteria, if applicable.
    • to Technology Providers describing in which circumstances which component of the Product has developed the erroneous behaviour.

Each set of verification reports is permanently stored in EGI's document database and made publicly available for review.

Staged-Rollout

Stage-Rollout complements the Quality Criteria Verification step in exposing the Product to real production conditions. Early Adopter sites that are interested in a timely rollout of the Product update take a part of their production infrastructure into a special "sandbox mode" where the Product update is installed under close scrutiny and exposed to a live environment and user requests.

The exact design and process of Staged-Rollout is described in the Operations context since [Staged-Rollout] needs close collaboration and planning within the Operations community.

Similar to Quality Criteria Verification, all Staged-Rollout activity is permanently stored in written reports per Product and will be publicly available for review and reference.

Software Provisioning

2nd line support: Distributed Middleware Support Unit

Glossary

Term Description
(UMD) Capability A Capability is scoped around well known and well-defined activities that are sufficiently disparate, independent, and fulfil a discrete task in a higher-level orchestrated sequence of actions taken by the user (manually or automated) in order to reach the aspired goal. Capabilities may depend on the presence of another Capability. Appliances, however, SHOULD NOT depend on each other to avoid technical dependencies and vendor lock-in.
Appliance An Appliance is the technical implementation of a Capability. An Appliance may be delivered in a single, monolithic Package, or it may be delivered as a set of components that are grouped into a set of Packages. In the latter case all packages must be installed to provide the complete Appliance.
Package A Package is a structured software unit suitable for automated installation on a computer. A Package may specify dependencies on other packages, so that either a specific version of that package, or a minimum version of that package may satisfy that dependency.
Product A Product is a solution delivered by Technology Providers to EGI and provides the functionality for one, sometimes more Capabilities as one single, undividable unit. Consequently, a Product may comprise of one or more Appliances.
Primary Package A Product consists of a well-defined set of specific packages, each in a specific version. Different versions (whether major, minor, or revision) of the same Product may share a subset of packages of the same version, that do not change between the compared versions.
Secondary Package A Secondary Package is a Package that is resolved from dependency declarations within a Package, that sources either from the Technology Provider's own Repository, or from any additional repository except OS distribution repositories. (Note: EPEL is an example of such a repository that is not an OS repository.)