AoD

From EGIWiki
Revision as of 15:24, 29 May 2017 by Larocca (talk | contribs) (Who can access the service ?)
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Applications on Demand Service menu: Home Documentation for providers Documentation for developers Architecture




The EGI Applications on Demand (also called AoD) service is the EGI’s response to the requirements of researchers, scattered across Europe, without dedicated access to computational and storage resources, as well as other facilities needed to run scientific applications.

The new service, through a lightweight user registration process, offers authorised users a grant, with a pre-defined quota of resources, which can be used to run a growing number of scientific applications from a portfolio. The grant to run scientific applications can be renewed or increased upon request. The portfolio of scientific applications is currently composed by a pre-defined set of applications from different scientific areas. This portfolio can be further extended thanks to the contributions of users of the service.


What does this service offer ?

The EGI Applications on Demand service is accessible through this User Registration Portal (URP) and through a lightweight identity vetting process offers:

  • Scientific applications that are offered ‘as services’ through graphical environments.
  • Science gateways and Virtual Research Environments hosting a portfolio of scientific applications. With these collaborative environments researchers can perform shared computations, and interact with High-Throughput and Cloud resources over the web.
  • Cloud and High-throughput compute resources suited for both compute/data intensive applications and for the hosting of scientific services.
  • Online Storage resources for storing scientific data that serve as input and output for computational jobs.
  • A network of Consultants and supporters who can provide guidance on the use of the service

Benefits

Through this service

  1. Researchers can access scientific applications underpinned by high-capacity compute and storage servers, to carry out compute and data-intensive simulations.
  2. Developers can integrate custom applications into the service and offer them as 'scalable, online services' for researchers and scientists worldwide.
  3. Providers of compute and storage services can share their clusters and clouds to serve 'the long tail of science'.

Already integrated components

The service operates as an open environment where any provider can integrate and share applications and compute/data components. The following components are already integrated into the service and are available for users to access:

Scientific Applications

AoDs_applications.jpg



Computer Science & Mathematics

  • The R Project for Statistical Computing (v3.2.2) is a language and environment for statistical computing and graphics.
    • The application is accessible through the Catania Science Gateway. The user manual is available here.
  • GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments.
    • The application is accessible through the EC3 portal.

Cultural Heritage

  • The Semantic Search Engine (SSE) is an application conceived to demonstrate the potential of information coupled with semantic web technologies to address the issues of data discovery and correlation.
    • The application is accessible through the Catania Science Gateway. The user manual is available here.

Life Sciences

  • Chispter is a user-friendly analysis software for high-throughput data. It contains over 300 analysis tools for next generation sequencing (NGS), microarray, proteomics and sequence data. Users can save and share automatic analysis workflows, and visualize data interactively using a built-in genome browser and many other visualizations.
    • The application is accessible through the Catania Science Gateway. The user manual is available here.
  • ClustalW2 is a multiple sequence alignment tool for the alignment of DNA or protein sequences.
    • The application is accessible through the Catania Science Gateway. The user manual is available here.
  • Galaxy is an open, web-based platform for data intensive biomedical research.
    • The application is accessible through the EC3 portal. Instructions on how to use Galaxy workflows with EC3 are available here.
  • AutoDock Vina is an open-source software for doing molecular docking.
    • The application is available through the WS-PGRADE/gUSE portal. The user guide to run the AutoDock Vina application is available here
  • NAMD is a parallel molecular dynamics code designed for high-performance simulation of large bio-molecular systems.
    • The application is available through the EC3 portal.

Others

  • Hello World is simple grid-based application that demonstrates the use of remote resources by printing the hostname where the job is executed.
    • The application is accessible through the Catania Science Gateway. The user manual is available here.
  • Apache Tomcat is an open source implementation of the Java Servlet, JavaServer Pages, Java Expression Language and Java WebSocket technologies. The Java Servlet, JavaServer Pages, Java Expression Language and Java WebSocket specifications are developed under the Java Community Process.
    • The application is accessible through the EC3 portal.
  • Gnuplot is a portable command-line driven graphing utility for Linux, OS/2, MS Windows, OSX, VMS, and many other platforms.
    • The application is accessible through the EC3 portal.
  • Docker is an open-source project that automates the deployment of Linux applications inside software containers.
    • The application is accessible through the EC3 portal.
  • Hadoop is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.
    • The application is accessible through the EC3 portal.
  • Marathon a production-grade container orchestration platform for Mesosphere's Data Center/Operating System (DC/OS) and Apache Mesos.
    • The application is accessible through the EC3 portal.
  • Chronos is a distributed and fault-tolerant scheduler that runs on top of Apache Mesos that can be used for job orchestration.
    • The application is accessible through the EC3 portal.
  • Jupyter is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text.
    • The application is accessible through the EC3 portal. Instructions on how to start Jupyter Notebook with EC3 are available here.

Science Gateways, Portals, Virtual Research Environments

  • The Catania Science Gateway (CSG) is a new generation of Science Gateway based on standard that changes the way e-Infrastructures are used. The gateway incorporates several scientific applications and offers these "as services" for the user. The CSG provides users with an intuitive web interface to execute applications on the grid resources of the platform. The CSG take cares of starting the job in one of the available resources of the platform, of transferring the needed files (e.g. executable, input files, etc.) and downloading the output(s) in behalf of the users. CSG is open source and released under the Apache 2.0 license.
    • The implementation of the CSG tailored for supporting the users of the long tail is available here
  • The WS-PGRADE Portal (Web Services Parallel Grid Runtime and Developer Environment Portal) is the Liferay-based web portal (WS-PGRADE web application) of gUSE, wich also includes a graphical portal service. WS-PGRADE is a Web portal hosted in a standard portal framework, using the client APIs of gUSE services to turn user requests into sequences of gUSE specific Web service calls. WS-PGRADE is integrated with the cloud resources of the platform, and provides a 'job wizard' interface for the user. Through the wizard one can define with a few clicks a computational job that WS-PGRADE will execute on the cloud resources. The environment takes care of instantiation of Virtual Machine image for the job, for sending input data for the job, and for the retrieval of job outputs.
    • The implementation of the WS-PGRADE/gUSE tailored for supporting the users of the long tail is available here
  • Elastic Cloud Computing Cluster (EC3) is a tool to create elastic virtual clusters on top of Infrastructure as a Service (IaaS) providers. EC3 uses 'recipes' to deploy TORQUE (optionally with MAUI) and SLURM clusters that can be self-managed with CLUES: it starts with a single-node cluster and working nodes will be dynamically deployed and provisioned to fit increasing load (number of jobs at the LRMS). Working nodes will be undeployed when they are idle. EC3 is integrated with the cloud resources of the platform, and provides a 'job wizard' interface for the user.
    • The implementation of the EC3 portal tailored for supporting the users of the long tail is available here

Who can access the service ?

The service is open for any researchers who needs a simple and user-friendly access to compute, storage and applications services in order to carry out data/compute intensive science and innovation. There are not any restrictions on the European organisations that can access and use the scientific applications integrated in this service. The researcher needs to be affiliated with, or at least have a partner (for example a referee), at a European research institution to qualify for access.

Through a lightweight vetting process, members of the EGI User Support team will perform the needed checks to validate the users' requests before to grant him/her the access to the large scale computing resources. This service is designed to meet the needs of individual researchers and small research groups who have limited or no experience with distributed and cloud computing.

Access requires acceptance of Acceptable Use Policy (AUP) and Conditions of Use of the 'EGI Applications on Demand Infrastructure'

How can you access the service ?

  1. Login to the User Registration Portal (URP) with the EGI CheckIn service.
  2. Setup a profile, including details about your affiliation and role within a research institute/project/team.
  3. Submit a resource request: indicate approximately how much computing and storage capacity you will underneath the scientific application of your choice. The default allocation is of 10TB of on-line storage and 1000h of CPUs, but you can ask for more resources too.
  4. After your request is approved, login to the gateway/portal environment that hosts the application you would like to use. If you are a developer you can also integrate custom applications with the compute and storage resources you are granted access to.

Help desk

For generic questions or inquiries, please contact the EGI Support team at applications-platform-support@mailman.egi.eu.

For gateway-specific support please contact:

  • Catania Science Gateway (CSG): sg-licence@ct.infn.it
  • WS-PGRAGE/gUSE portal: portalwide@lpds.sztaki.hu
  • EC3 portal: micafer1@upv.es

How to acknowledge this service

Users of the service are asked to provide appropriate acknowledgement of the use in scientific publications. The following acknowledgement text can be used for this purpose:

This work used the EGI Applications On Demand Service, which is co-funded by the EGI-Engage project (Horizon 2020) under Grant number 654142.

Further reading

Presentations about the service:

  • Overview of the EGI Infrastructure for serving the long tail (EGI Community Forum, November 2015): [1]
  • Poster and animated slides from Demo at EGI Community Forum, November 2015 (Winner of best demo prize): [2]
  • Slideset about the concept of the EGI long-tail of science platform (from Nov. 2014): [3]
  • Slideset about the authentication and authorization model adopted (from Nov. 2015): [4]
  • Slideset about the staus report of the platform at the EGI Conference 2016: [5]
  • Slideset about the staus report of the EGI platform at the DI4R Conference 2016: [6]