Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "UMDQualityCriteria Generic"

From EGIWiki
Jump to navigation Jump to search
Line 243: Line 243:
'''Description'''
'''Description'''


UMD Software must be available in different packaging formats
UMD Software must be available in the packaging formats of the supported OS.


'''Pass/Fail Criteria'''
'''Pass/Fail Criteria'''


Pass if software is available at least in the following formats:
For all software:
* TAR/ZIP or similar source distribution
* Provide a tar.gz/zip or similar distribution
* RPM
 
* DEB
For RPM based OS:
* Provide RPM packages with correct dependencies
 
For DEB based OS:
* Provide DEB packages with correct dependencies.
 
The supported OSes is determined by the Portability QC of each component.

Revision as of 10:27, 25 October 2010

Documentation

Services in UMD must include a comprehensive documentation written in a uniform and clear style. All Quality Criteria described below may be met by a single document that contains all the requested sections.

Functional description

Description

Document with a functional description of the component

Verification

Existence and quality of the document

Pass/Fail criteria

The document must exist and contain basic information about the component

Release notes

Description

Document with the release notes of the component

Verification

Existence and quality of the document

Pass/Fail criteria

The release notes document must exist and contain most relevant information about the release

User documentation

Description

User guide describing the functionality of the software and how to use it

Verification

Check availability of user guide. Check completeness of user guide.

Pass/Fail criteria

User guide must exist and contain information about the usage.

Man pages

Description

Commands should include documentation on its usage and options. It may not be applicable to services without user interface commands

Verification

Availability and quality of man pages.

Pass/Fail criteria

If the component provides command line tools, they must have man pages. Man pages should contain information about the usage. If man pages are not available, comprehensive help options must be included with the command with information about the usage.

API Documentation

Description If service has an API, include documentation of all its functions.

Verification

Availability and quality of API documentation.

Pass/Fail criteria

If the component provides an API, documentation must exist and cover all the functions of the API. For each function, there must be a description of its functionality and its arguments.

Administrator Documentation

Descrpiton

Administrator guide describing installation, configuration and operation of the system

Verification

Availability and completeness of the guide.

Pass/Fail criteria

Installation guide must exist. Configuration and operation documents for services must also exist.

Service Reference Card

Description

For each of the services that the component runs, include the following documentation:

Service Reference Card
Service ServiceName
Description Description of the service
Init scripts List of init scripts for the service, expected run levels
Daemons List of daemons needed for the service
Configuration List of configuration files used by the service
Logs List of log files used by the service
Open ports List of ports the service uses
Cron List of crons used by the service
Any other relevant information Include here any other relevant information about the service

Pass/Fail Criteria

All the services needed by a component must have the previous table with complete information.

Source Code Quality

The source code of each component of the UMD middleware should follow a coherent and clear programming style that helps in the readability of the code and eases maintenance, testing, debugging, fixing, modification and portability of the software. Open source components must publicly offer their source code and the license with the binaries.

License

Description

All UMD components must have a compatible license for using it in the EGI infrastructure

Verification

Check the component license

Pass/Fail criteria

TBD: Compatible licenses in EGI ?

Source Code Availability

Description

Open source code must provide pointers to the source

Verification

Check existence of source code

Pass/Fail Criteria

If the component is open source, source must be available for download.

Build Procedures

Description

Open Source components should provide a documented and reproducible build procedure

Verification

Check existence of build documentation and existence of automatic build procedures.

Pass/Fail Criteria

Build documentation for open source components must exist. Automatic build procedures are recommendable.

Management, Monitoring, Traceability

UMD components should have mechanisms for managing them, monitoring their status and tracing actions they perform on the system. Ideally, these should be also available remotely, allowing operators to react timely to problems in the infrastructure. TBD: Is there any standard interface for this?

Service control and status

Description

Services run by the component must provide a mechanism for starting, stopping and querying the status of the services following the typical init scripts conventions as documented in http://refspecs.freestandards.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html

Verification

Check the start/stop mechanism for the service. Test all possible combinations:

  • start with already started services
  • start with services stopped
  • stop with stopped services
  • stop with started services
  • check status with started services
  • check status with stopped services

Pass/Fail criteria

Service must provide mechanisms for starting and stopping services. They must work properly in all the cases described above.

Service responsiveness

Descripition

Services must react to requests timely. They should provide a mechanism to check their responsiveness to the requests

Verification

Provide a test that checks the service responds to requests (e.g. expected ports are open, expected answer to requests is received).

Pass/Fail criteria

Test for service responsiveness must be available and executed correctly.

Service monitoring

Description

All components in the EGI testbed must provide monitoring probes that can be executed automatically by a monitoring framework such as Nagios. The probes should check the service responsiveness and correctness (good replies for typical requests).

Documentation for the current monitoring framework for EGI is available at: TBD ?

Particular monitoring probes are defined at the Specific Quality Criteria for each component. Here a list of generic probes that must be provided are listed.

Certificate lifetime

Description

Provide a monitoring probe that test the host certificate lifetime for the service is valid. This QC is only applicable to services that need a host certificate.

Input

Host to check.

Ouput

OK if certificate is valid. ERROR if certificate is not valid.

Pass/Fail Criteria

Pass: probe exists and calculate the certificate lifetime correctly. Probe can integrated in the monitoring framework.

Fail: if the probe does not exist or is not able to get the certificate lifetime or it cannot be integrated in the monitoring framework.

Traceability

Description

All actions taken by the service should be traceable by the service administrator. Log files or similar mechanisms should be provided by the service.

Verification

Provide a test that checks that the tracing mechanism of the service is working properly (e.g. log file of the service is generated)

Pass/Fail Criteria

Test for the tracing mechanism must exist. It must be executed correctly.

Software Packaging

Description

UMD Software must be available in the packaging formats of the supported OS.

Pass/Fail Criteria

For all software:

  • Provide a tar.gz/zip or similar distribution

For RPM based OS:

  • Provide RPM packages with correct dependencies

For DEB based OS:

  • Provide DEB packages with correct dependencies.

The supported OSes is determined by the Portability QC of each component.