Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

VT VAPOR:VAPOR OpsPortal Engage

From EGIWiki
Jump to navigation Jump to search

User community management

The User Database implemented in the Operations Portal retrieves user information from the VOMS servers. It is proposed to extend this database to be a one-stop repository for (i) information about grid and cloud users including users behind a robot certificate, and (ii) the management of user registration life-cycle. This user community management solution is dedicated primarily to VO managers. (à voir de votre côté à qui vous dédiez d'abord ce type de service?)

The User Database is intended to store user information besides the administrative data available in the VOMS. The following information is envisaged:

  • Administrative data (DN, email, affiliation, VO membership, group membership) in sync with the VOMS database.
  • Research discipline classification referring to the VT Scientific Discipline Classification.
  • Scientific publications: keep track of published works using the infrastructure and encourage users to acknowledge the usage of EGI resources.
  • Robot certificate DN (if any) in order to have a realistic idea of the number of actual users in a VO (may users have their own personal grid certificate or not).
  • Scientific applications used linking to the EGI Applications Database.
  • User File Catalog instance (if any) and base directory.

As much as possible, the life cycle workflow automates interactions with third party services such as the VOMS, file catalog, EGI Applications Database, mailing list system. At registration time, the user is automatically requested to provide details about their activity, affiliation and the applications they use (link to the EGI Applications Database). A VO administrator approves or reject pending requests. On approval, the user's root directory is created in the file catalog. After membership expiration, files of expired users are cleaned up after a grace period.

Collaborations may be considered with the Perun user management system for the user life-cycle workflow management, and the VT Scientific Publications Repository Implementation Resources and OpenAIRE project to collect scientific publications based on resource usage acknowledgement.

Infrastructure resources monitoring and views

Currently, the Oeprations Portal and VAPOR provide complementary tools and views dedicated to either resource centres or VOs:

  • The Operations Portal provides a resource distribution browser and dashboard both dedicated to resource centre administrators (il faut sans doute ajouter un peu plus de détails).
  • VAPOR provides views dedicated to VO managers and support teams:
    • The VO resource view provides a consolidated view of resources supporting a VO by crossing GOCDB and BDII data, and a specific view of resource not in production.
    • Monitor the number of running vs. waiting jobs at each site with additional statistical reports.
    • View of job success/error/time-out rates at each site, compute a white-list of best-performing sites from the VO perspective.
    • VO Data Management features perform file catalog-based scan of storage elements (to monitor storage feeling up, handle decommissioning...), and consistency checks (cleanup of dark data/lost files).


It is proposed to extend the scope of both tools to cloud resources while integrating VAPOR more closely into the Operations Portal. Thus the Operations Portal will become a one-stop tool for dashboards and views of resource, either from the resource centre perspective of from the VO perspective.

  • Operations Portal (j'ai copié ça depuis les slides de notre présentation commune, je te laisse compléter/consolider avec le reste):
    • Extends the resource distribution browser to cloud resources, give more details (OS , number of cores , capacity), replace some Gstat feature, provide an API.
    • Extend the dashboard to monitor cloud infrastructures.
  • VAPOR:
    • Extend the VO resource view to report the status and characteristics of cloud resource providers.
    • Monitor the number of running VMs vs. number of VM creation requests at each cloud site.
    • Monitor the VM creation success/error/time-out rates at each cloud site, compute a white-list of best-performing cloud sites from the VO perspective.
    • Extend VO Data Management features to deal with storage available for a VO at cloud sites, add support for file catalog solutions that will be adopted by the EGI Federated Cloud.