Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

OMB Requirements

From EGIWiki
Revision as of 13:29, 10 December 2014 by Krakow (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security
Alert.png This article is Deprecated and should no longer be used, but is still available for reasons of reference.


Summary of requirements collected by EGI Operations

List of requirements by EGI Operations from the end of 2010 to February 2011. 

Every ticket is briefly described, with a summary of the feedback from the technology providers if any. For a more exhaustive description of the request, and to read the discussion that followed the ticket submission, follow the ticket link.

Note: This is a static page, last update: 31-JAn-2012 16:30


Installability

RT ticket number 2563
Requirements title Mandatory variables in configuration files should be clearly identified
Requirement description Middleware services configurations files (e.g. YAIM configuration file) has always a set of variable that *must* be edited in any installation of the service. While there are variables that can stay as they are, if particular customizations are not needed.

In the configuration file templates the variables to be modified should be well identified with a string like: VAR_TO_CHANGE = TO-BE-CHANGED Configuration scripts should check that *all* the variables so identified were edited by the service administrator.

Comment from Technology provider [1]
Answer(s) from EGI/requestor
Status submitted



RT ticket number 1357
Requirements title Middleware use standard file locations
Requirement description Currently services create a huge amount of temporary files and logs, not all of them are in standard locations. Services should create all the logs files under /var/logs, and the temporary files under /tmp.

Having all the files under the same path makes easier to parse and collect the logs.

Comment from Technology provider EMI: Most of the EMI services already comply with the requirement, nevertheless covered by EMI objective, will be included in the EMI-2 plans.
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 1379
Requirements title Better configuration of services using mysql backend.
Requirement description Many services need a custom configuration of the database backend to improve their performance. This configuration should be provided with the service, and automatically applied where possible.
Comment from Technology provider The requirement will be part of EMI-2 workplan.
Answer(s) from EGI/requestor comment
Status endorsed



RT ticket number 1022
Requirements title UI dependencies on python 2.4
Requirement description Middleware components should get rid of obsolete dependencies, that could be not available in more recent s.o.
Comment from Technology provider It seems that the main problem is Ubuntu, EMI will make available UI on ubuntu.
Answer(s) from EGI/requestor Requester asked to check if also other components have those obsolete deps.
Status clarification



RT ticket number 1186
Requirements title New features for target system
Requirement description New features for Target System: Support for clusters without a shared FS, Possibility to define resources offered by a site per queue in IDB (assuming full support for queue selection, Possibility to freely split IDB into multiple files. Except of better manageability this would allow us to easy and automatically update some of SSR supported by a site.
Comment from Technology provider EMI required more information. Task:[2]
Answer(s) from EGI/requestor comment
Status planned



RT ticket number 1388
Requirements title Better documentation on service replication and load balancing configurations
Requirement description Middleware providers should provide documentation on how to easily migrate stateful services from a machine to another (e.g. how to migrate databases and data files). Documentation on high availability configuration is also needed.
Comment from Technology provider EMI will not take a general requirement on documentation. Where available HA configurations descriptions should be already provided being part of the documentation policy of EMI. Task: [3]
Answer(s) from EGI/requestor comment
Status planned



RT ticket number 1381
Requirements title Improve YAIM configuration
Requirement description YAIM functions should automatically get as many parameters as possible. E.g. operating system and architecture, needed in the cluster configuration, should be configured automatically.
Comment from Technology provider YAIM is not fully supported by EMI
Answer(s) from EGI/requestor comment
Status rejected


Portability

RT ticket number 1202
Requirements title Uniform logging format
Requirement description Middleware services should uniform their logging format.
Comment from Technology provider EMI won't endorse this requirement. Changing the log format would break a number of monitoring tools that parse the logs. In case of specific problems open a GGUS ticket.
Answer(s) from EGI/requestor At least there should be some common rules to avoid common problems in parsing the logs. Single ticket will be opened to report specific logging problems.
Status on hold



RT ticket number 1201
Requirements title Homogeneity in service control
Requirement description Requirement to have a standard, uniform, way to start/stop services.
Comment from Technology provider EMI endorsed this requirement as part of EMI-2 technical plan.
Answer(s) from EGI/requestor comment
Status endorsed



Usability

RT ticket number 2652
Requirements title Support multi-VO services(WMS), association to multiple topBDIIs
Requirement description This requirement refers to a scenario where there are Top-BDII containing subset of services and sites (for example belonging to different projects), and the possibility to use a single WMS to submit to all the subset (even if different communities - most probably - would submit to different subsets).
Comment from Technology provider [4]GGUS ticket
Answer(s) from EGI/requestor
Status endorsed (by PT)



RT ticket number 2280
Requirements title Support for LSF
Requirement description The requirement is to keep LSF supported by all CE implementation, in particular by BLAH and CREAM.
Comment from Technology provider
Answer(s) from EGI/requestor
Status on hold



RT ticket number 1380
Requirements title Better WMS configuration tools, and monitoring.
Requirement description Many configuration/setup tricks are in the manual pages of WMS, but not yet implemented in the automatic configuration of the services. From the max number of connections to automatically clean the ICE persist directory.
Comment from Technology provider Open a GGUS ticket to the product team.
Answer(s) from EGI/requestor GGUS ticket for this requirement is created: https://ggus.eu/ws/ticket_info.php?ticket=71173
Status released



RT ticket number 1184
Requirements title Brokering and Matchmaking support for new site info system types
Requirement description UNICORE: Use of SSr in brokering process, and extensions to service orchestrator.
Comment from Technology provider EMI does not support those components.
Answer(s) from EGI/requestor Comment accepted, request forwarded directly to the developers, ticket solved
Status solved



RT ticket number 1282
Requirements title Requirement for VOMS/VOMSAdmin and for the accounting systems
Requirement description Requirement for the possibility to force a user to choose a VO-group at the moment of VO registration. This is needed for catch-all VOs, where the VO groups identify the actual user research community.
Comment from Technology provider EMI requested clarifications, before the endorsement. Task: [5]
Answer(s) from EGI/requestor The simplest and probably the most useful scenario is the following: among the available VOs there is the possibility to choose which ones have the "mandatory flag" enabled
Opened GGUS ticket: https://ggus.eu/tech/ticket_show.php?ticket=76685
Status planned



RT ticket number 1382
Requirements title Documentation on different implementation of the same service
Requirement description The technology providers should provide comparison tables for the different implementation of the same service.
Comment from Technology provider EMI endorsed the requirement. Final document expected by EMI-2 release. Task: [6]
Answer(s) from EGI/requestor
Status planned



RT ticket number 1203
Requirements title Uniform API
Requirement description Request to have a uniform set of APIs to control all the services.
Comment from Technology provider EMI does provide an uniform API interface for all the services, single uniform api may be delivered for the single areas.
Answer(s) from EGI/requestor
Status rejected


Operability

RT ticket number 1236
Requirements title Generic information system for EMI
Requirement description Requirement to have an information system (BDII in gLite, GIIS in ARC) middle ware independent. Especially, packages provided to build up a server on such a system must be independent of any middle ware components.
Comment from Technology provider EMIR will provide the possibility to have a common information system.
Answer(s) from EGI/requestor
Status planned



RT ticket number 2699
Requirements title Globus middleware publishes accounting data
Requirement description Globus middleware should publish accounting data to the EGI.eu repositories. The accounting records for grid jobs submitted through Globus must be published using an agreed messaging infrastructure and format.
Comment from Technology provider
Answer(s) from EGI/requestor
Status submitted



RT ticket number 2695
Requirements title Argus publishes itself in the information system
Requirement description ARGUS does not publish itself in the information discovery system. Even if it is a local service, the publishing of ARGUS is necessary for operational reasons, in order to monitor its deployment status in the production infrastructure.
Comment from Technology provider This requirement is already addressed by a development task.
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 1674
Requirements title Unify SRMv2.2 port
Requirement description Different implementation of the SRM service are answering on different ports (dCache implementation and DPM). In general the same service should provide the same default port in all its implementations.
Comment from Technology provider [7]
Answer(s) from EGI/requestor
Status rejected



RT ticket number 2677
Requirements title VOMS publishes number of users
Requirement description VOMS should expose methods that return the number of users and the number of suspended users, registered in that VOMS service, for suspended users more detailed information might be needed, like, for example: number of users suspended for security reasons or expired membership. Some of these figures should be available in the information system to easily collect them.
Comment from Technology provider Open GGUS ticket
Answer(s) from EGI/requestor GGUS ticket opened
Status delivered


Job Management

RT ticket number 3262
Requirements title SLURM support in CREAM
Requirement description This requirement is supported by the output collected from a survey targeted to sites deploying CREAM. According to the feedback received, 39 Sites will consider to replace their current LRMS with SLURM if supported by CREAM. These sites in total contribute 64600 cores (17% of the job slots available in the infrastructure). 23 of these contribute individually more than 1k cores.
Comment from Technology provider
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 1235
Requirements title Batch systems supported by EMI
Requirement description The EMI computing elements should support a larger set of batch systems, the support (at least for the more important LRMS) should be guaranteed for the future. Another request was to have a common set of batch systems explicitly supported by all the Middleware stacks.
Comment from Technology provider (It is not a requirement directly addressed to EMI) EMI endorsed the requirement, but more clarifications are needed, starting from a list of needed batch systems needed/required by sites.
Answer(s) from EGI/requestor A survey on the batch systems deployed/wanted has been launched by EGI in May-June 2011
Status on_hold



RT ticket number 1283
Requirements title MPI-start improvement
Requirement description Currently MPI-start does not support hybrid MPI-openMPI application and cpu/memory affinity between worker nodes involved in the process.
Comment from Technology provider Product team released the features in EMI-1 Update 8 . Task [8]
Answer(s) from EGI/requestor
Status delivered



RT ticket number 1385
Requirements title Interactive job monitoring
Requirement description Request for more tools to monitor runnign interactive jobs.
Comment from Technology provider EMI requested more details, and a more specific requirement. task [9]
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 1386
Requirements title EMI Data clients should be able to offer the file:// protocol to SRM services
Requirement description Storm should support the direct access file protocol
Comment from Technology provider Storm server already supports the file:// protocol. The endorsed request is to enable it in the clients. [10]
Answer(s) from EGI/requestor comment
Status planned


Usability

RT ticket number 2652
Requirements title Support multi-VO services, association to multiple topBDIIs
Requirement description description
Comment from Technology provider
Answer(s) from EGI/requestor
Status waiting answers



RT ticket number 1378
Requirements title Publishing middleware service versions
Requirement description All the services should publish in the information system the service version, and the middleware release. Other additional request, automatically gathered from the operating system, should also be published: o.s., architecture..
Comment from Technology provider EMI endorsed the request, more information about what and how will be published are needed.
Answer(s) from EGI/requestor A proposal about how to map these information on GLUE2 has been forwarded to EMI
Status planned



RT ticket number 1177
Requirements title Archaic DRMAA interface version enabled in glite-TORQUE_server breaks functionality
Requirement description Old dependencies are breaking some functionalities of glite_TORQUE
Comment from Technology provider Submit bug report.
Answer(s) from EGI/requestor Ticket RESOLVED
Status delivered



RT ticket number 1183
Requirements title UNICORE: Site info system enhancements
Requirement description UNICORE info system should support String, Integer, Enum types for SSR. And information on the expected waiting time of the job in the queue.
Comment from Technology provider EMI endorsed the requirement, fill a GGUS ticket with the requirement. [11]
Answer(s) from EGI/requestor
Status planned (partially)



RT ticket number 1383
Requirements title cluster information
Requirement description The information described by the glite-CLUSTER compoent should be implemented in CREAM and in other implementation of computing capacity.
Comment from Technology provider Requested more details
Answer(s) from EGI/requestor
Status delivered



RT ticket number 1188
Requirements title GUI enhancements
Requirement description Requirement for enhancements in the UNICORE GUI
Comment from Technology provider EMI does not endorse the requirement, because the component is not in EMI
Answer(s) from EGI/requestor
Status rejected


Scalability

RT ticket number 3279
Requirements title The BDII must scale with number of sites.
Requirement description Two main scaling issues have been identified:
  • The number of sites, that with the current implementation corresponds to the number of threads which top-bdii runs to contact the site-bdiis.
  • The total amount of information in the top-bdii - which is increasing more than proportionally with the number of sites - lower the freshness of the information published.

Something more that just a performance improvement is needed which just puts the limit up to new threshold unless that threshold can be increased by a multiple 10 or so.

Comment from Technology provider [12]
Answer(s) from EGI/requestor
Status waiting


Serviceability

RT ticket number 3278
Requirements title Enable automatic VO membership renewal
Requirement description VOMS should be configurable in order to automatically renew the VO membership when a user renew the AUP acceptance. In order to remove the VO Manager's action.

If the VO Manager wants to keep the user membership renewal on himself, he must be warned with an e-mail in advance, before the user's membership expires.

Comment from Technology provider [13]
Answer(s) from EGI/requestor
Status under evaluation



RT ticket number 2538
Requirements title Quick identification of users affected in an SE intervention
Requirement description To make easier to warn files owner in case of SE planned maintenance, there should be a single-line command, to get the list of all the users who own the files contained in the SEs. This is currently possible, but it would be nice to have a single command to get the information.
Comment from Technology provider [14]
Answer(s) from EGI/requestor
Status planned
(partially)



RT ticket number 1983
Requirements title Error messages from middleware
Requirement description Error messages quality need to be enhanced to make easier the troubleshooting. Main points are: a) making a statement on the priority of good error messages, eg how much priority fixing error-message bugs should have. b) some standardization of exit codes may be useful. Look to see what has been done by others and whether it is applicable. c) issues dealing with how messages propagate from lower levels to upper levels. This has been frequently cited by MW people : "if the lower level does not trap the error there is nothing we can do about it".

The first middleware component to be improved is LCAS/LCMAPS

Comment from Technology provider
Answer(s) from EGI/requestor
Status on hold
too generic



RT ticket number 881
Requirements title Requirement for VO renaming / migration
Requirement description The first part of the requirements is to implement a mechanism to allow users to migrate their data between the VOs they are member of. This would also easy the VO renaming process.
The second part of the requirement is to extend the authorization and authentication mechanism to the VO aliases (defined by OPS portal).
Comment from Technology provider EMI security team reported that the requirement is not well specified, and not clear.
Answer(s) from EGI/requestor Requester reported a use case: Users (and VO administrators) are interested in migrating data from one VO1 to another VO2 in a transparent way. This should be done without doubling the space needed for the data, and without downloading/uploading the files.
Status Under discussion internally by Tech providers



RT ticket number 1189
Requirements title Server-side management tools
Requirement description UNICORE need management tools for: easy identification of orphaned job directories; possibility to convert old state DB to the newer server; current activity status of the server; ability to disable job submission. Easy way to trace a user's job for support purposes.
Comment from Technology provider [15]
Answer(s) from EGI/requestor
Status planned


Availability

RT ticket number 3268
Requirements title APEL monitoring does not raise alarm when CE stops to publish
Requirement description APEL tests in nagios may not detect all cases in which the site does not publish accounting data. The problem affects sites with more than one CE. The monitoring should be improved by the rule that each production CE should publish the data.
Comment from Technology provider
Answer(s) from EGI/requestor
Status endorsed


RT ticket number 2279
Requirements title Load balancing for CREAM and LFC
Requirement description CREAM and LFC should support load balancing
Comment from Technology provider EMI accepts the ticket because:FTS 3 includes loadbalancing as part og ist planned features, CREAM will consider introducing load balancing on a longer term [16]
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 2278
Requirements title CREAM, ARGUS and Myproxy High-Availability
Requirement description CREAM, ARGUS and Myproxy should support high availability.
Comment from Technology provider Argus: [17]
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 1181
Requirements title glite-BDII hangs, service status reports "bdii OK"
Requirement description The service status for BDII does not report hanging statuses.
Comment from Technology provider Fill bug report.
Answer(s) from EGI/requestor
Status delivered



RT ticket number 1384
Requirements title DOS attacks
Requirement description Services should be able to protect themselves against misuses. E.g. a dCache server can be easily killed by a huge amount of legal requests submitted simultaneously by the same user.
Comment from Technology provider Task: [18]. EMI makes sure that each service comes with some sort of configurable resource protection. However, general protection against DOS is not possible.
Answer(s) from EGI/requestor
Status clarification



RT ticket number 1607
Requirements title accounting monitoring must be sufficient to decide whether all deployed middlewares of a site publish (correct) accounting data
Requirement description If a site deploys more than one middleware, the monitoring of accounting should check if all the computing elelment of the different stacks are publishing their accounting data. The same problem could be raised about multiple CE in the same site.
Comment from Technology provider
Answer(s) from EGI/requestor This ticket was a duplication of another ticket.
Status endorsed


File Access

RT ticket number 1187
Requirements title Some new features for Data Mgmt and Sharing
Requirement description UNICORE. Support for end users collaboration, manipulating permssions, logical SMS
Comment from Technology provider EMI does not support that UNICORE component.
Answer(s) from EGI/requestor
Status tech provider internal clarification



RT ticket number 1673
Requirements title SRMv1 to be removed from gLite DPM box
Requirement description SRMv1 is not used anymore on DPM. It could be removed
Comment from Technology provider Not submitted yet.
Answer(s) from EGI/requestor
Status delivered



Performance

RT ticket number 1173
Requirements title Increase of LB WS interface performance
Requirement description LB performance decrease when the WS interface is used.
Comment from Technology provider EMI endorsed the requirement, but it is low priority
Answer(s) from EGI/requestor
Status delivered



RT ticket number 1182
Requirements title glite-CREAM consumes a lot of memory
Requirement description After about a month of uptime CREAM-CE performance decrease because of the memory usage.
Comment from Technology provider Task: [19] EMI endorsed the requirement in the objective: "Increase performance of EMI services", that will be in the EMI-2 tech. plan.
Answer(s) from EGI/requestor
Status rejected


Security

This set of requirements - concerning security, authentication and authorization topics - have been proposed and discussed within IGTF and EUGridPMA. They were not directly discussed by OMB but by EGI security bodies representatives.

RT ticket number 3080
Requirements title RPDNC constrains
Requirement description The middleware MUST provide a way to enforce Relying-Party Defined Namespace constraints, and SHOULD use a single format for expressing such RPDNCs.
Comment from Technology provider [20]
Answer(s) from EGI/requestor comment
Status endorsed



RT ticket number 3079
Requirements title Default key size for any generated proxy
Requirement description All generated proxy should have a default key size >= 1024 bits. If proxy life span is longer than ten (10) days the key size should increase to 2048.
Comment from Technology provider [21]
Answer(s) from EGI/requestor comment
Status endorsed



RT ticket number 3078
Requirements title Support for SHA2 proxies
Requirement description All authentication libraries must support also proxies signed with SHA2 family algorithms.
Comment from Technology provider [22]
Answer(s) from EGI/requestor comment
Status endorsed



RT ticket number 3076
Requirements title Support for OCSP
Requirement description All the middleware services that need to validate certificates/proxies should support the Online Certificates Status Protocol, as a more secure and configurable alternative to CLRs.
Comment from Technology provider [23]
Answer(s) from EGI/requestor
Status endorsed



RT ticket number 3081
Requirements title Support drop-in trust anchor distribution
Requirement description All the middleware services must support drop-in trust anchor distribution for x.509 trust anchors, and this must be maintained also in the future releases.
Comment from Technology provider [24]
Answer(s) from EGI/requestor comment
Status endorsed



RT ticket number 3075
Requirements title Common Authentication Library to configure the accepted proxy lifetime
Requirement description At site level should be possible to configure the services in order to reject proxies with a life time longer than a defined time.
Comment from Technology provider [25]
Answer(s) from EGI/requestor comment
Status endorsed



RT ticket number 3074
Requirements title Unit Test for CRL refersh
Requirement description CLRs must be enabled to update directly in the file system, and services must poll the directory to check for updates without the need to restart the service. All the services using CRLs All must have a test unit/integration test to check this feature is implemented, and that new releases do not break this behavior.
Comment from Technology provider [26]
Answer(s) from EGI/requestor comment
Status endorsed