Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Applications on Demand Service - architecture

From EGIWiki
Jump to navigation Jump to search
Applications on Demand Service menu: Home Documentation for providers Documentation for developers Architecture




Applications on Demand (AoD) Service Information pages

The EGI Applications on Demand service (AoD) is the EGI’s response to the requirements of researchers who are interested in using applications in a on-demand fashion together with the compute and storage environment needed to compute and store data.

The Service can be accessed through the EGI Marketplace

Service architecture

The EGI Applications on Demand service architecture is presented in Figure.


AoDs_architecturev2.jpg


This service architecture is composed by the following components:

  • A catch-all VO called ‘vo.access.egi.eu’ and a pre-allocated pool of HTC and cloud resources configured for supporting the EGI Applications on Demand research activities. This resource pool currently includes cloud resources from Italy (INFN-Catania), Spain (CESGA) and France (IN2P3-IRES).
  • The X.509 credentials factory service (also called eToken server)[1] is a standard-based solution developed for central management of robot certificates and the provision of Per-User Sub-Proxy (PUSP) certificates. PUSP allows to identify the individual user that operates using a common robot certificate. Permissions to request robot proxy certificates, and contact the server, are granted only to Science Gateways/portal integrated in the Platform and only when the user is authorized.
  • A set of Science Gateways/portal to host applications that users can run accessing the EGI Applications on Demand service. Currently the following three frameworks are used: The FutureGateway Science Gateway (FGSG), the WS-PGRADE/gUSE portal and the Elastic Cloud Computing Cluster (EC3).
  • The EGI VMOps dashboard provides a Graphical User Interface (GUI) to perform Virtual Machine (VM) management operations on the EGI Federated Cloud. The dashboard introduces a user-friendly environment where users can create and manage VMs along with associated storage devices on any of the EGI Federated Cloud providers. It provides a complete view of the deployed applications and a unified resource management experience, independently of the technology driving each of the resource centres of the federation. Users can create new infrastructure topologies, which include a set of VMs, their associated storage and contextualization, a wizard-like builder that guides them through the selection of the virtual appliances, virtual organisation, resource provider, and the final customisation of the VMs that will be deployed. Its tight integration with the AppDB Cloud Marketplace allows for an automatic discovery of the appliances which are supported at each resource provider Once a topology has been created, VMOps allows management actions to be applied both on the set of VMs comprising a topology and on fine-grained actions on each individual VM.
  • The EGI Notebooks, build on top of JupyterHub offers an open-source web application where users can create and share documents that contain live code, equations, visualization and explanatory text.


[1] Valeria Ardizzone, Roberto Barbera, Antonio Calanducci, Marco Fargetta, E. Ingrà, Ivan Porro, Giuseppe La Rocca, Salvatore Monforte, R. Ricceri, Riccardo Rotondo, Diego Scardaci, Andrea Schenone: The DECIDE Science Gateway. Journal of Grid Computing 10(4): 689-707 (2012)

Available resources

Current available resources grouped by categories:

  • Cloud Resources:
    • 246 vCPU cores
    • 781 GB of RAM
    • 2.2TB of object storage
  • High-Throughtput Resources:
    • ~13Million HEPSPEC
    • 1.4 TB of disk storage

For more details about the resources allocated for supporting this service, please click here.

Operational Level Agreements


GGUS Support Units (SUs)

  • FutureGateway Science Gateway (FGSG):
    • The FGSG GGUS SU is available under the Core Services category
  • Elastic Cloud Computing Cluster (EC3):
    • The EC3 GGUS SU is available under the Core Services category
  • WS-PGRADE/gUSE portal:
    • The WS-PGRADE/gUSE GGUS SU is available under the Core Services category
  • The EGI Notebooks:
    • The EGI Notebooks GGUS SU is available under the EGI Notebooks category
  • The EGI VMOps dashboard:
    • The AppDB GGUS SU is available under the Core Services category


Links for administrators

User approval:


Monitoring

  • The EGI Applications on Demand service components have been registered in the GOCDB and connected with the EGI monitoring system based on ARGO.
  • By default ARGO automatically gathered the services endpoints from the GOCDB and implements simple 'https' checks using standard NAGIOS probes. If necessary new ones can be easily developed and added.
  • The following service components are monitored by ARGO:
    • EGI-FGSG,
    • EGI-NOTEBOOKS,
    • GRIDOPS-WS-PGRADE,
    • GRIDOPS-APPDB, and
    • GRIDOPS-EC3
  • To monitor the service components, check the following ARGO report page.


Accounting

  • Accounting data about the VO users can be checked here: https://accounting.egi.eu/
  • From the EGI Accounting Portal it is possible to check the accounting metrics generated for both grid- and cloud-based resources supporting the vo.access.egi.eu VO.
  • From the top-menu click on 'Restrict View' and 'VO Admin' to check the accounting data of platform users.


EGI_AoDs_Accounting.png

Policies & documents


Useful Links


Roadmap

  • Integration of the JupyterHub as a Service (mid 2018)
  • Integration of the Galaxy as a Service
  • Extend the portfolio of applications adopting the COntinuous DElivery of Research Applications in a Distributed Environment (CODE-RADE) [1] approach (mid 2018)
  • Investigate the integration of the Ophidia framework for supporting data-intensive analysis (2019)
  • Upgrade of the CSG to Liferay 7 to use the Future Gateway (FG) API server developed in the context of the INDIGO-DataCloud project (2019)
  • Integrate REPAST, a family of free agent-based modeling libraries
  • Integration the open-source serverless Computing Platform for Data-Processing Applications
  • Improve the user's experience in the EC3 portal (mid 2018)
  • Integrate the HNSciCloud voucher schemes in the EC3 portal (2018)
  • Joined the IN2P3-IRES cloud provider in the vo.access.egi.eu VO (2019)

[1] https://github.com/SouthAfricaDigitalScience