UMDQualityCriteria Generic
Documentation
Services in UMD must include a comprehensive documentation written in a uniform and clear style. All Quality Criteria described below may be met by a single document that contains all the requested sections.
Functional description
Description
Document with a functional description of the component
Verification
Existence and quality of the document
Pass/Fail criteria
The document must exist and contain basic information about the component
Release notes
Description
Document with the release notes of the component
Verification
Existence and quality of the document
Pass/Fail criteria
The release notes document must exist and contain most relevant information about the release
User documentation
Description
User guide describing the functionality of the software and how to use it
Verification
Check availability of user guide. Check completeness of user guide.
Pass/Fail criteria
User guide must exist and contain information about the usage.
Man pages
Description
Commands should include documentation on its usage and options. It may not be applicable to services without user interface commands
Verification
Availability and quality of man pages.
Pass/Fail criteria
If the component provides command line tools, they must have man pages. Man pages should contain information about the usage. If man pages are not available, comprehensive help options must be included with the command with information about the usage.
API Documentation
Description If service has an API, include documentation of all its functions.
Verification
Availability and quality of API documentation.
Pass/Fail criteria
If the component provides an API, documentation must exist and cover all the functions of the API. For each function, there must be a description of its functionality and its arguments.
Administrator Documentation
Descrpiton
Administrator guide describing installation, configuration and operation of the system
Verification
Availability and completeness of the guide.
Pass/Fail criteria
Installation guide must exist. Configuration and operation documents for services must also exist.
Service Reference Card
Description
For each of the services that the component runs, include the following documentation:
Service | ServiceName |
---|---|
Description | Description of the service |
Init scripts | List of init scripts for the service, expected run levels |
Daemons | List of daemons needed for the service |
Configuration | List of configuration files used by the service |
Logs | List of log files used by the service |
Open ports | List of ports the service uses |
Cron | List of crons used by the service |
Any other relevant information | Include here any other relevant information about the service |
Pass/Fail Criteria
All the services needed by a component must have the previous table with complete information.
Source Code Quality
The source code of each component of the UMD middleware should follow a coherent and clear programming style that helps in the readability of the code and eases maintenance, testing, debugging, fixing, modification and portability of the software. Open source components must publicly offer their source code and the license with the binaries.
License
Description
All UMD components must have a compatible license for using it in the EGI infrastructure
Verification
Check the component license
Pass/Fail criteria
TBD: Compatible licenses in EGI ?
Source Code Availability
Description
Open source code must provide pointers to the source
Verification
Check existence of source code
Pass/Fail Criteria
If the component is open source, source must be available for download.
Build Procedures
Description
Open Source components should provide a documented and reproducible build procedure
Verification
Check existence of build documentation and existence of automatic build procedures.
Pass/Fail Criteria
Build documentation for open source components must exist. Automatic build procedures are recommendable.
Management, Monitoring, Traceability
UMD components should have mechanisms for managing them, monitoring their status and tracing actions they perform on the system. Ideally, these should be also available remotely, allowing operators to react timely to problems in the infrastructure. TBD: Is there any standard interface for this?
Service control and status
Description
Services run by the component must provide a mechanism for starting, stopping and querying the status of the services following the typical init scripts conventions as documented in http://refspecs.freestandards.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html
Verification
Check the start/stop mechanism for the service. Test all possible combinations:
- start with already started services
- start with services stopped
- stop with stopped services
- stop with started services
- check status with started services
- check status with stopped services
Pass/Fail criteria
Service must provide mechanisms for starting and stopping services. They must work properly in all the cases described above.
Service responsiveness
Descripition
Services must react to requests timely. They should provide a mechanism to check their responsiveness to the requests
Verification
Provide a test that checks the service responds to requests (e.g. expected ports are open, expected answer to requests is received).
Pass/Fail criteria
Test for service responsiveness must be available and executed correctly.
Service monitoring
Description
All components in the EGI testbed must provide monitoring probes that can be executed automatically by a monitoring framework such as Nagios. The probe should check the service responsiveness and correctness (good replies for typical requests). Documentation for the current monitoring framework for EGI is available at: TBD ?
Verification
Check the existence of the component probe and check its integration in Nagios
Pass/Fail criteria
Probe must exist and must be executed correctly in the monitoring framework.
Traceability
'Description
All actions taken by the service should be traceable by the service administrator. Log files or similar mechanisms should be provided by the service.
Verification
Provide a test that checks that the tracing mechanism of the service is working properly (e.g. log file of the service is generated)
Pass/Fail Criteria
Test for the tracing mechanism must exist. It must be executed correctly.