Galaxy workflows with EC3

Overview

For users

For resource providers

Infrastructure status

Site-specific configuration

Architecture

Galaxy workflows with EC3

Introduction

This guide is intended for researchers who want to use Galaxy, an open web-based platform for data-intensive research, in the cloud-based resources provided by the EGI Long Tail of Science Platform.

Objectives

In this guide we will show how to:

• Deploy an elastic cluster of VMs in the RECAS-BARI resource of the EGI Federation
• Install a Galaxy web-based platform for data intensive research with a Torque Server as backend
• Install CLUE as elasticity manager
• Install custom Galaxy tool (booties2)
• Download and register in Galaxy the Escherichia Coli Genome

Virtual Elastic Cluster: components

The Elastic Cloud Computing Cluster (EC3) is a framework to create elastic virtual clusters on top of Infrastructure as a Service (IaaS) providers is composed by the following components:

• Infrastructure Manager (IM) is a tool that eases the access and the usability of IaaS clouds by automating the VMI selection, deployment, configuration, software installation, monitoring and update of Virtual Appliances. It supports APIs from a large number of virtual platforms, making user applications cloud-agnostic. In addition it integrates a contextualization system to enable the installation and configuration of all the user required applications providing the user with a fully functional infrastructure.

• Resource and Application Description Language (RADL) is a tool used to specify the requirements of the resources where the scientific applications will be executed. It must address not only hardware (CPU number, CPU architecture, RAM size, etc.) but also software requirements (applications, libraries, data base systems, etc.). It should include all the configuration details needed to get a fully functional and configured VM (a Virtual Appliance or VA). It merges the definitions of specifications, such as OVF, but using a declarative scheme, with contextualization languages such as Ansible. It also allows describing the underlying network capabilities required.

• CLUE is an energy management system for High Performance Computing (HPC) Clusters and Cloud infrastructures. The main function of the system is to power off internal cluster nodes when they are not being used, and conversely to power them on when they are needed. CLUES system integrates with the cluster management middleware, such as a batch-queuing system or a cloud infrastructure management system, by means of different connectors.

• EC3 as a Service (EC3aaS) is a web service offered to the community to facilitate the usage of EC3 to non-experienced users. Anyone can access the website and try the tool by using the user-friendly wizard to easily configure and deploy Virtual Elastic Clusters on multiple Clouds.

Create Galaxy workflows on the LToS platform

Galaxy, web-based platform for data-intensive research, is one of the applications available in the EGI LToS platform. The platform is accessible through this portal and offers grid, cloud and application services from across the EGI community for individual researchers and small research teams.

Configuration and Deployment of a Cluster

To configure and deploy a Virtual Elastic Cluster using EC3aaS, a user accesses the front page and clicks on the "Deploy your cluster!" link as shown in figure: