Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI Quality Criteria Release 2"

From EGIWiki
Jump to navigation Jump to search
(Created page with '== QC Release Notes == These release notes describe the first release of the Quality Criteria for UMD software that is expected to be delivered in early August 2011. The docume…')
 
 
(18 intermediate revisions by the same user not shown)
Line 1: Line 1:
== QC Release Notes ==
{{Tech menubar}}
{{Tech QA submenu}}
{{TOC_right}}


These release notes describe the first release of the Quality Criteria for UMD software that is expected to be delivered in early August 2011.
These release notes describe the second release of the Quality Criteria for UMD software delivered in early August 2011.


The documents can be found at [https://documents.egi.eu/public/ShowDocument?docid=364 DocDB #364], current status is DRAFT.
The documents can be found at [https://documents.egi.eu/public/ShowDocument?docid=364 DocDB #364], current status is FINAL.


 
== What's New ==
=== What's New ===
Several changes have been introduced in the documents, the most relevant ones are:
Several changes have been introduced in the documents, the most relevant ones are:
* New template more compact and with uniform formatting of fields.
* New (more compact) template and with uniform formatting of fields.
* Collected
* New capabilities covered: Messaging, Client API, Interactive Job, Remote Instrumentation
 
* Clarification of criteria especially on criterion applicability and pass/fail description.
 
* Merge similar criteria into more complete ones (See especially Compute Capabilities Criteria).
=== Detailed changes ===
 
* Requirements from User Community
* Requirements from Operations Community (especially software incidents found in production)


* Deficiencies in criteria found in Verification or Stage Rollout
== Documents ==
** Update in GENERIC_REL_5, GENERIC_REL_4
The Quality Criteria (QC) can be found in the [https://documents.egi.eu/public/ShowDocument?docid=364 DocDB #364], there is one document for each type of capability:
 
* Recommendations and issues found by the Software Vulnerability Group
** New criteria: GENERIC_SEC_2
 
* Analysis of reference implementations of UMD Capabilities defined in the Roadmap
** New capabilities covered: Messaging, Client API, Remote Instrumentation & Interactive Job Management.
 
* Review and analysis of feedback from Technology Providers
** Clarification and removing ambiguous descriptions of criteria (Several criteria affected)
** IGE: new criteria GENERIC_REL_6 & GENERIC_REL_7. Update of GENERIC_DOC_5
** EMI: new template
 
   
   
   
   
   
 
=== Documents ===
The Quality Criteria (QC) can be found in the [https://documents.egi.eu/public/ShowDocument?docid=240 QC Specification document], there is one document for each type of capability:


* Generic QC, for all software
* Generic QC, for all software
Line 50: Line 27:
Take into account that a software product may cover QC specified in more than one of those documents.
Take into account that a software product may cover QC specified in more than one of those documents.


=== Deprecated QC ===
== Covered Capabilities ==
 
None
 
=== Covered Capabilities ===
From all capabilities identified in version 2 of [https://documents.egi.eu/document/272 UMD Roadmap], the following are covered:
From all capabilities identified in version 2 of [https://documents.egi.eu/document/272 UMD Roadmap], the following are covered:
==== Security Capabilities ====
=== Security Capabilities ===
* Authentication
* Authentication
* Attribute Authority (using [https://twiki.cern.ch/twiki/bin/view/EMI/VOMS VOMS] as reference implementation)
* Attribute Authority (using [https://twiki.cern.ch/twiki/bin/view/EMI/VOMS VOMS] as reference implementation)
Line 62: Line 35:
* Credential Management (using [http://grid.ncsa.illinois.edu/myproxy/ MyProxy] as reference implementation)
* Credential Management (using [http://grid.ncsa.illinois.edu/myproxy/ MyProxy] as reference implementation)


==== Information Capabilities ====
=== Information Capabilities ===
* Information Model
* Information Model
* Information Discovery
* Information Discovery
* Messaging


==== Storage Capabilities ====
=== Storage Capabilities ===
* File Encryption/Decryption (using [https://twiki.cern.ch/twiki/bin/view/EGEE/DMEDS Hydra] as reference implementation)
* File Encryption/Decryption (using [https://twiki.cern.ch/twiki/bin/view/EGEE/DMEDS Hydra] as reference implementation)
* File Access
* File Access
Line 73: Line 47:
* Storage Management
* Storage Management


==== Data Capabilities ====
=== Data Capabilities ===
* Data Access (using [http://www.ogsadai.org.uk/ OGSA-DAI] as reference implementation)
* Data Access (using [http://www.ogsadai.org.uk/ OGSA-DAI] as reference implementation)
* Metadata Catalogue (using [https://twiki.cern.ch/twiki/bin/view/EMI/AMGA AMGA] and [https://twiki.cern.ch/twiki/bin/view/LCG/DataManagementTop#LFC_and_DPM LFC] as reference implementations)
* Metadata Catalogue (using [https://twiki.cern.ch/twiki/bin/view/EMI/AMGA AMGA] and [https://twiki.cern.ch/twiki/bin/view/LCG/DataManagementTop#LFC_and_DPM LFC] as reference implementations)


==== Compute Capabilities ====
=== Compute Capabilities ===
* Job Execution
* Job Execution
* Parallel Job
* Parallel Job
* Interactive Job Management
* Job Scheduling (using [http://web.infn.it/gLiteWMS/ WMS] as reference implementation)
* Job Scheduling (using [http://web.infn.it/gLiteWMS/ WMS] as reference implementation)


==== Operations Capabilities ====
=== Client API Capability ===
* Client API
 
=== Remote Instrumentation ===
* Remote Instrumentation
 
=== Operations Capabilities ===
* Monitoring  
* Monitoring  
* Accounting (using [https://twiki.cern.ch/twiki/bin/view/EMI/APELClient APEL] as reference implementation)
* Accounting (using [https://twiki.cern.ch/twiki/bin/view/EMI/APELClient APEL] as reference implementation)


=== Non Covered Capabilities ===
== Non Covered Capabilities ==
Several capabilities identified in the UMD Roadmap are still missing the Quality Criteria due to lack of a clear reference implementation or missing requirements from the EGI community. Most of the missing capabilities are introduced in the last UMD roadmap version.
Some capabilities identified in the UMD Roadmap are still missing the Quality Criteria due to lack of a clear reference implementation or missing requirements from the EGI community. Most of the missing capabilities are introduced in the last UMD roadmap version.


* Messaging
* Interactive Job Management
* Workflow
* Workflow
* Virtual Machine Management
* Virtual Machine Management
* Virtual Machine Image Format
* Virtual Machine Image Format
* Image Distribution
* Image Distribution
* Remote Instrumentation
* Client API


The Quality Criteria Task will continue the process of defining the missing capabilities for the next release of the QC.
The Quality Criteria Task will continue the process of defining the missing capabilities for the next releases of the QC.
 
== Detailed Changelog ==
 
* Requirements from User Community
** WMS Stability and Perfomance: JOBSCH_SERVICE_3, JOBSCH_SERVICE_4
**  #1385: INTERACTIVE_JOB_3
** #1386: FILEACC_API_1
 
* Requirements from Operations Community (especially software incidents found in production)
** No incidents found in operations (no previous releases of UMD)
** Review of monitoring probes (MON_PROBE_*)
** #1357: GENERIC_REL_4
** #2274: GENERIC_SERVICE_1
 
* Deficiencies in criteria found in Verification or Stage Rollout
** Update in GENERIC_REL_5, GENERIC_REL_4
** Removal of JOBSCH_WMS_API_2
 
* Recommendations and issues found by the Software Vulnerability Group
** New criteria: GENERIC_SEC_2, JOBSCH_WMS_SEC_1
 
* Analysis of reference implementations of UMD Capabilities defined in the Roadmap:
** New capabilities covered: Messaging (MSG_*), Client API (CLIENT_*), Remote Instrumentation (INSTRUMENT_*) & Interactive Job Management (INTERACTIVE_*)
** Review of existing criteria: AUTHZ_PDP_1, METADATA_LFC_FUNC_5
** New criteria for Storage Appliances (STORAGE_DEVICE_2-4)
 
* Review and analysis of feedback from Technology Providers
** Clarification and removing ambiguous descriptions of criteria (several criteria affected)
** IGE: new criteria GENERIC_REL_6, GENERIC_REL_7, GENERIC_SERVICE_4. Update of GENERIC_DOC_5. Update of AUTHN_IFACE_1.
** EMI: new template, better description of compute criteria JOBEXEC_*, JOBSCH_*, PARALLEL_*. Review of scalability/reliability criteria. See also EGI_Quality_Criteria_Release_2_EMI_Review with detailed changes taken as part of EMI review of QC documents draft version.

Latest revision as of 12:56, 5 August 2011

Technology Software Component Delivery Software Provisioning UMD Middleware Cloud Middleware Distribution Containers Distribution Technology Glossary


Quality Assurance | Quality Criteria Definition | Quality Criteria Dissemination | Quality Criteria Verification | Verifier Guideline | Verification Testbed | Glossary





These release notes describe the second release of the Quality Criteria for UMD software delivered in early August 2011.

The documents can be found at DocDB #364, current status is FINAL.

What's New

Several changes have been introduced in the documents, the most relevant ones are:

  • New (more compact) template and with uniform formatting of fields.
  • New capabilities covered: Messaging, Client API, Interactive Job, Remote Instrumentation
  • Clarification of criteria especially on criterion applicability and pass/fail description.
  • Merge similar criteria into more complete ones (See especially Compute Capabilities Criteria).

Documents

The Quality Criteria (QC) can be found in the DocDB #364, there is one document for each type of capability:

  • Generic QC, for all software
  • Security Capabilities QC
  • Information Capabilities QC
  • Storage Capabilities QC
  • Data Capabilities QC
  • Compute Capabilities QC
  • Operations Capabilities QC

Take into account that a software product may cover QC specified in more than one of those documents.

Covered Capabilities

From all capabilities identified in version 2 of UMD Roadmap, the following are covered:

Security Capabilities

  • Authentication
  • Attribute Authority (using VOMS as reference implementation)
  • Authorization (using Argus as reference implementation)
  • Credential Management (using MyProxy as reference implementation)

Information Capabilities

  • Information Model
  • Information Discovery
  • Messaging

Storage Capabilities

  • File Encryption/Decryption (using Hydra as reference implementation)
  • File Access
  • File Transfer
  • File Transfer Scheduling (using FTS as reference implementation)
  • Storage Management

Data Capabilities

  • Data Access (using OGSA-DAI as reference implementation)
  • Metadata Catalogue (using AMGA and LFC as reference implementations)

Compute Capabilities

  • Job Execution
  • Parallel Job
  • Interactive Job Management
  • Job Scheduling (using WMS as reference implementation)

Client API Capability

  • Client API

Remote Instrumentation

  • Remote Instrumentation

Operations Capabilities

  • Monitoring
  • Accounting (using APEL as reference implementation)

Non Covered Capabilities

Some capabilities identified in the UMD Roadmap are still missing the Quality Criteria due to lack of a clear reference implementation or missing requirements from the EGI community. Most of the missing capabilities are introduced in the last UMD roadmap version.

  • Workflow
  • Virtual Machine Management
  • Virtual Machine Image Format
  • Image Distribution

The Quality Criteria Task will continue the process of defining the missing capabilities for the next releases of the QC.

Detailed Changelog

  • Requirements from User Community
    • WMS Stability and Perfomance: JOBSCH_SERVICE_3, JOBSCH_SERVICE_4
    • #1385: INTERACTIVE_JOB_3
    • #1386: FILEACC_API_1
  • Requirements from Operations Community (especially software incidents found in production)
    • No incidents found in operations (no previous releases of UMD)
    • Review of monitoring probes (MON_PROBE_*)
    • #1357: GENERIC_REL_4
    • #2274: GENERIC_SERVICE_1
  • Deficiencies in criteria found in Verification or Stage Rollout
    • Update in GENERIC_REL_5, GENERIC_REL_4
    • Removal of JOBSCH_WMS_API_2
  • Recommendations and issues found by the Software Vulnerability Group
    • New criteria: GENERIC_SEC_2, JOBSCH_WMS_SEC_1
  • Analysis of reference implementations of UMD Capabilities defined in the Roadmap:
    • New capabilities covered: Messaging (MSG_*), Client API (CLIENT_*), Remote Instrumentation (INSTRUMENT_*) & Interactive Job Management (INTERACTIVE_*)
    • Review of existing criteria: AUTHZ_PDP_1, METADATA_LFC_FUNC_5
    • New criteria for Storage Appliances (STORAGE_DEVICE_2-4)
  • Review and analysis of feedback from Technology Providers
    • Clarification and removing ambiguous descriptions of criteria (several criteria affected)
    • IGE: new criteria GENERIC_REL_6, GENERIC_REL_7, GENERIC_SERVICE_4. Update of GENERIC_DOC_5. Update of AUTHN_IFACE_1.
    • EMI: new template, better description of compute criteria JOBEXEC_*, JOBSCH_*, PARALLEL_*. Review of scalability/reliability criteria. See also EGI_Quality_Criteria_Release_2_EMI_Review with detailed changes taken as part of EMI review of QC documents draft version.
Retrieved from "https://wiki.egi.eu/w/index.php?title=EGI_Quality_Criteria_Release_2&oldid=22977"