Galaxy workflows with EC3

Overview

For users

For resource providers

Infrastructure status

Site-specific configuration

Architecture

Galaxy workflows with EC3

Introduction

This guide is intended for researchers who want to use Galaxy, an open web-based platform for data-intensive research, in the cloud-based resources provided by the EGI Long Tail of Science Platform.

Objectives

In this guide we will show how to:

• Deploy an elastic cluster of VMs in the RECAS-BARI resource of the EGI Federation
• Install a Galaxy web-based platform for data intensive research with a Torque Server as backend
• Install CLUE as elasticity manager
• Install custom Galaxy tool (booties2)
• Download and register in Galaxy the Escherichia Coli Genome

Virtual Elastic Cluster: components

The Elastic Cloud Computing Cluster (EC3) tool to create elastic virtual clusters on top of Infrastructure as a Service (IaaS) providers is composed by the following components:

• Infrastructure Manager (IM) is a tool that eases the access and the usability of IaaS clouds by automating the VMI selection, deployment, configuration, software installation, monitoring and update of Virtual Appliances. It supports APIs from a large number of virtual platforms, making user applications cloud-agnostic. In addition it integrates a contextualization system to enable the installation and configuration of all the user required applications providing the user with a fully functional infrastructure.

• Resource and Application Description Language (RADL) is a tool used to specify the requirements of the resources where the scientific applications will be executed. It must address not only hardware (CPU number, CPU architecture, RAM size, etc.) but also software requirements (applications, libraries, data base systems, etc.). It should include all the configuration details needed to get a fully functional and configured VM (a Virtual Appliance or VA). It merges the definitions of specifications, such as OVF, but using a declarative scheme, with contextualization languages such as Ansible. It also allows describing the underlying network capabilities required.

• CLUE is an energy management system for High Performance Computing (HPC) Clusters and Cloud infrastructures. The main function of the system is to power off internal cluster nodes when they are not being used, and conversely to power them on when they are needed. CLUES system integrates with the cluster management middleware, such as a batch-queuing system or a cloud infrastructure management system, by means of different connectors