EGI Quality Criteria Release 2

From EGIWiki
Revision as of 17:22, 4 August 2011 by Enolfc (talk | contribs)
Jump to: navigation, search
Technology Software Component Delivery Software Provisioning UMD Middleware Cloud Middleware Distribution Containers Distribution Technology Glossary

Quality Assurance | Quality Criteria Definition | Quality Criteria Dissemination | Quality Criteria Verification | Verifier Guideline | Verification Testbed | Glossary

These release notes describe the second release of the Quality Criteria for UMD software delivered in early August 2011.

The documents can be found at DocDB #364, current status is FINAL.

What's New

Several changes have been introduced in the documents, the most relevant ones are:

  • New (more compact) template and with uniform formatting of fields.
  • New capabilities covered: Messaging, Client API, Interactive Job, Remote Instrumentation
  • Clarification of criteria especially on criterion applicability and pass/fail description.
  • Merge similar criteria into more complete ones (See especially Compute Capabilities Criteria).


The Quality Criteria (QC) can be found in the DocDB #364, there is one document for each type of capability:

  • Generic QC, for all software
  • Security Capabilities QC
  • Information Capabilities QC
  • Storage Capabilities QC
  • Data Capabilities QC
  • Compute Capabilities QC
  • Operations Capabilities QC

Take into account that a software product may cover QC specified in more than one of those documents.

Covered Capabilities

From all capabilities identified in version 2 of UMD Roadmap, the following are covered:

Security Capabilities

  • Authentication
  • Attribute Authority (using VOMS as reference implementation)
  • Authorization (using Argus as reference implementation)
  • Credential Management (using MyProxy as reference implementation)

Information Capabilities

  • Information Model
  • Information Discovery
  • Messaging

Storage Capabilities

  • File Encryption/Decryption (using Hydra as reference implementation)
  • File Access
  • File Transfer
  • File Transfer Scheduling (using FTS as reference implementation)
  • Storage Management

Data Capabilities

  • Data Access (using OGSA-DAI as reference implementation)
  • Metadata Catalogue (using AMGA and LFC as reference implementations)

Compute Capabilities

  • Job Execution
  • Parallel Job
  • Interactive Job Management
  • Job Scheduling (using WMS as reference implementation)

Client API Capability

  • Client API

Remote Instrumentation

  • Remote Instrumentation

Operations Capabilities

  • Monitoring
  • Accounting (using APEL as reference implementation)

Non Covered Capabilities

Some capabilities identified in the UMD Roadmap are still missing the Quality Criteria due to lack of a clear reference implementation or missing requirements from the EGI community. Most of the missing capabilities are introduced in the last UMD roadmap version.

  • Workflow
  • Virtual Machine Management
  • Virtual Machine Image Format
  • Image Distribution

The Quality Criteria Task will continue the process of defining the missing capabilities for the next releases of the QC.

Detailed Changelog

  • Requirements from User Community
    • WMS Stability and Perfomance: JOBSCH_SERVICE_3, JOBSCH_SERVICE_4
    • #1385: INTERACTIVE_JOB_3
    • #1386: FILEACC_API_1
  • Requirements from Operations Community (especially software incidents found in production)
    • No incidents found in operations (no previous releases of UMD)
    • Review of monitoring probes (MON_PROBE_*)
    • #1357: GENERIC_REL_4
    • #2274: GENERIC_SERVICE_1
  • Deficiencies in criteria found in Verification or Stage Rollout
    • Update in GENERIC_REL_5, GENERIC_REL_4
    • Removal of JOBSCH_WMS_API_2
  • Recommendations and issues found by the Software Vulnerability Group
    • New criteria: GENERIC_SEC_2, JOBSCH_WMS_SEC_1
  • Analysis of reference implementations of UMD Capabilities defined in the Roadmap:
    • New capabilities covered: Messaging (MSG_*), Client API (CLIENT_*), Remote Instrumentation (INSTRUMENT_*) & Interactive Job Management (INTERACTIVE_*)
    • Review of existing criteria: AUTHZ_PDP_1, METADATA_LFC_FUNC_5
  • Review and analysis of feedback from Technology Providers
    • Clarification and removing ambiguous descriptions of criteria (several criteria affected)
    • IGE: new criteria GENERIC_REL_6, GENERIC_REL_7, GENERIC_SERVICE_4. Update of GENERIC_DOC_5. Update of AUTHN_IFACE_1.
    • EMI: new template, better description of compute criteria JOBEXEC_*, JOBSCH_*, PARALLEL_*. Review of scalability/reliability criteria. See also EGI_Quality_Criteria_Release_2_EMI_Review with detailed changes taken as part of EMI review of QC documents draft version.