Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

VT VAPOR

From EGIWiki
Jump to navigation Jump to search

General Project Information

  • Leader: Franck Michel, CNRS
  • Mailing List:
  • Meetings: <link to Indico container>
  • Status: In progress
  • Start Date: 15/04/2013
  • Duration: 12 months
  • Customer: Virtual Organizations


Project description

Overview

This project intends to help small to medium-size grid user communities perform daily administrative and operational tasks, by developing VAPOR, the Vo Administration and operations PORtal, a generic tool to assist community managers and support teams in performing their daily activities. Such communities may typically have no or few dedicated IT support, have scattered scientific activities or fragmented user groups, and may possibly (although not necessarily) make an opportunistic usage of the resources.

This portal is expected to

  • help such communities sustain their model by mutualising the administrative and operational cost at the level of a Virtual Research Community (VRC) or beyond,
  • facilitates the outreach of new user communities by making it easier to start with the administration and operations of a VO. This portal will be available to any community wishing to deploy it, and licensed as open source software.

The primary use case of this project will be the Life-Science Grid Community (LSGC), starting with the biomed Virtual Organization (VO), and mutualising this activity with other VOs of the LSGC.

Motivation

Operating large subsets of the European Grid Infrastructure (EGI) with reasonable quality of service requires substantial effort to cope with runtime issues like hardware failures or configuration flaws. While this cost is acceptable for large and structured communities such as High Energy Physics backed with a solid IT support, it is often out of reach for small VOs and fragmented communities. The Virtual Research Community (VRC) model has encouraged such communities to organize and structure themselves, as illustrated by the Life-Science VRC (LSGC) that gathers five VOs related to the Life-Science field.

Among them, the biomed VO has set up a volunteering-based technical support team, which goal is to be the technical liaison between the VO users, the resources providers, and the EGI instances (UCB, OMB, UCST). Most active users of the VO contribute their time and experience to perform this job on a volunteering basis, and relay each other during duty shifts. However, the two-years experience of this team shows that the effort required to operate such a VO remains high even when mutualised with members the VO, and it is most likely unreachable for new emerging communities.

Finally, this experience suggests that, to be sustainable, the volunteering model should be mutualised not only within the VO, but also with other VOs, in order to reduce its cost while improving its efficiency.

Some communities require operation and administration tools tailored to their model

A large variety of existing tools and portals (SAM, VO Admin Dashboard, VO Operations Portal, VOMS) are available to assist VO managers and support teams. While some are very generic, others are designed to meet somewhat specific contexts. An example is the CERN Experiment Dashboard, which, although generic and customizable, is tailored for large VOs with common needs, and does not easily fit in the context of smaller "opportunistic" VOs with heterogeneous user groups.

The insight gained in biomed attests that mutualising the effort at the VRC level still requires efforts. In particular, some administration and operational tasks need to be facilitated or automated in different areas: community users management, community-wide accounting, data management, quality of service (QoS) monitoring. Incidentally, the VRC model having no technical existence in the middleware, VRCs lack tools able to provide "VRC-aware" views. From these observations, in 2011 the LSGC members agreed on the need for a tool primarily tailored to the LSGC VOs, though generic enough to meet needs of other similar communities.

Motivation

project mandate - who has directed the formation of the project, what has been defined as the project objective. Who is the project leader/manager, who are the stake holders (this is vitally important as you need to be clear on what it is that is expected of you), who are the recipients of the project output and when is this required by. Describe the formation of the project team (how why and where), the allotted resources (time, people, stuff, money etc). There could be a number of sub paras. You should also make clear that the scope of the task is realistic and that you can achieve it – as such, complete this introduction at the start of the project and re-check it periodically as you progress.

Objectives

VAPOR is a generic portal, independent of scientific applications, designed to assist VO managers and VO support teams in their daily administration and operational tasks.

VAPOR complements existing tools (SAM, VO Admin Dashboard, VO Operations Portal, VOMS) with novel features tailored to such communities, hereafter sorted in three categories:

  • Community users management:
    • Users database, handling and follow-up of user registration lifecycle
    • Keep track of real users "hidden behind" a robot certificate
    • Interface with external services: VOMS, LFC, EGI Applications Database
    • Collection of publications and scientific production achieved using the infrastructure
    • Maintain VRC-wide and VO-wide mailing lists in sync with VOMS
    • Operations management for technical support teams:
    • Resources status indicators & statistical reports: (i) issue reports on resources availability (free storage space, jobs success rate), (ii) report GOCDB and BDII status to exclude resources not in production
  • Data Management: files migration before storage decommissioning or in case of SE filling-up, clean-up files of former users
    • Accounting:
    • On-demand generation of reports about global community resource usage, per VO resource usage, per VO sub-group resource usage
    • Projection of future needs for storage and computing resources

Tasks

List and describe the specific tasks with target dates for completion.

Outcomes/Deliverables

  • Deliverable 1 description and target date
  • Deliverable 2 description

Members

  • Member 1 Name and Role in team
  • Member 2 Name and Role in team
  • Stake holder 1 and reason for interest
  • Stake holder 1 and reason for interest


Emerging Information

if relevant - may change the nature and objectives of the project


Resources

How is the Project Team to be resourced and how will members work? How much effort will be required from each individual, how will this effort impact contributing organisations? – do not confuse VT participants with stakeholders. VT participants do work for the project – others don’t have to do any work for the project. How will your funds be consumed?


Progress Reporting

How will you measure your progress and how will you report this? To whom? The objective here is to assure others that your project is on track, but if there are problems then this is the path to getting help and more resources (time, funding, effort).

The Project Leader will provide a short emailed progress report on a weekly basis. The report will be due by 17:00 on Fridays and is to contain details of: ● Work achieved that week ● Work planned for next week ● Progress against the goals in the project plan ● Issues that the virtual team leader needs help with (e.g. non-responsive partners, more resources, support from EGI.eu teams, etc.)

The Team project manager will also provide a more comprehensive input for the EGI InSPIRE quarterly reporting.