Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Federated Cloud Communities

From EGIWiki
Revision as of 23:13, 12 October 2014 by Nunolf (talk | contribs)
Jump to navigation Jump to search
Main Roadmap and Innovation Technology For Users For Resource Providers Media



This page provides detailed information about the user communities and use cases who are using/integrating the EGI Federated Cloud services.

Fedclouduclogos.png


Current FedCloud Users and Communities

This table provides a summary of the communities that are already collaborating with the EGI Federated Cloud offering use cases, requirements and valuable feedback.

Project or Application Description Use Cases Further information Contacts
INERTIA INERTIA project addresses the "structural inertia" of existing Distribution Grids by introducing more active elements combined with the necessary control and distributed coordination mechanisms. To this end INERTIA will adopt the Internet of Things/Services principles to the Distribution Grid Control Operations.
  • Use Case 1: Selection of the optimal portfolio on an emergency grid operation - Congestion issues are related with the power that flows among the lines or the transformers of a power system. The power flows among the different network’s equipment must be maintained within acceptable operational limits (equipment’s thermal limits) in order to prevent equipment failures. In emergency situation a selection of optimal local hub can be used to keep the grid stable.
  • Use Case 2: Selection of the optimal portfolio on a non emergency grid operation - This use case describes the situation where the total Consumption of an Aggregated portfolio should be optimized based on the reception of data from market operations. This can be the prediction of tariff information, the operation of Reserve Markets.
  • Use Case 3: Monitoring of energy data (Aggregator Hub installation) - A rich monitoring tool is the prerequisite to determine state and performance of the total hub portfolio. It has to collect measured data on different time-scales from DERs, Building Energy Management Systems (Local Hubs) and also manage the regulation of data provision with a large variety of graphical data representations. Different monitoring approaches are delivered for the available data depending on the time horizon.
  • Use Case 4: Setting User Preferences - Final Occupants will have the capability of explicitly setting their preferences through a personalized Ambient UI. Moreover, they will have full control over DERs within their personalised working area, using traditional controls (light switches, HVAC panels etc.). If they feel that their preferences are violated, any automated control will be postponed, informing the system of the corrective action performed by the user and the potential discomfort caused.
FedCloudINERTIA
  • EGI: Diego Scardaci (diego.scardaci@egi.eu)
  • Engineering: Giampaolo Fiorentino (giampaolo.fiorentino@eng.it), Luigi Briguglio (luigi.briguglio@eng.it)
Jena Adaptable Modelling System (JAMS) JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams).

JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week.

  • Use Case: run JAMS on the EGI Federated Cloud
FedCloudJAMSPoC
  • JAMS: Christian Fischer (christian.fischer.2@uni-jena.de) - Department of Geoinformatics, Hydrology and Modelling of Friedrich Schiller University Jena
  • EGI: Diego Scardaci (diego.scardaci@egi.eu)
Peachnote Peachnote.com is a music score search engine and analysis platform. The system is the first of its kind and can be thought as an analog of Google Books Ngram Viewer and Google Books search for music scores. Peachnote provides visitors and researchers access to a massive amount of symbolic music data.
  • Use Case 1: the ability to upload and start a prepared VMware VM. The VM has only to be able to make outbound connections: to Amazon's SQS for job information, to HBase cluster to retrieve and store data, and to the peachnote server to regularly update the workflow code. No inbound connections are needed, which hopefully means less administrative and security concerns.
  • Use Case 2: the ability to run a small Hadoop and HBase cluster in the cloud.
FedCloudPeachnote
  • EGI: Gergely Sipos (gergely.sipos@egi.eu), Salvatore Pinto (salvatore.pinto@egi.eu)
  • Peachnote: Vladimir Viro (vladimir@viro.name)
Physiome project Project of web application to compute, store and provide values of parameters of physiological models via lightweight API. The current version support model calibration. Further development will focus on storing the calibrated model parameters along with the data of real experiments and simulations and metadata of models. See slides presented at the EGI CF 2014 for more info.
  • Use Case: Run on the EGI Federated Cloud a system to support modeling of physiological systems in the phase of calibrating model parameters and in the phase of simulating different scenarios. The loosely coupled part of the system is deployed in a remote distributed computational capacity. The system capabilities is accessible via a web application and allows user to focus on experimental data, names of parameters, visual control of the calibration computation and hide unnecessary complexity of the remote subsystems computation. The data of real experiments and simulations are stored and provided for further research.
FedCloudPhysiome
  • EGI: Diego Scardaci (diego.scardaci@egi.eu), Nuno Ferreira (nuno.ferreira@egi.eu)
  • Physiome: Tomas Kulhanek (tomas.kulhanek@lf1.cuni.cz)
READemption - A RNA-Seq Analysis Pipeline RNA-sequencing analysis of data has performed to understand different biological features and behaviour. The research team generated a workflow based on a pipeline built to satisfy many experiments using Python 3.4, third party-libraries (numpy, pysam, matplotlib) and some external tools as segemehl and R. The running time heavily depends on the size and type of data. Usually takes some hour to some day. The workflow will be published as open source tool soon. Indeed, it’s a very common approach.
  • Use Case: the use case consist to run this workflow on the EGI Federated Cloud.
FedCloudRNASeq
  • EGI: Diego Scardaci (diego.scardaci@egi.eu), Salvatore Pinto (salvatore.pinto@egi.eu)
  • Core Unit Systems Medicine - University of Wurzburg: Konrad Foster
TRUFA - Transcriptomes User-Friendly Analysis TRUFA (Transcriptomes User-Friendly Analysis) is a free webserver designed to help researchers in genomics to perform de novo RNA-seq analysis. TRUFA aims to be a public service targeting a broad audience, namely comparative biology, biomedicine, bioinformatics and others. Sequencing methods are becoming cheaper and thus it is foreseeable that researchers studying models as well as non-models organisms could make use of this service. Having such pipeline ready for use will allow any biologist to perform RNA-seq analysis in a user-friendly way.
  • Use Case: Exploiting FedCloud resources from the TRUFA Portal
FedCloudTRUFA
  • EGI: Diego Scardaci (diego.scardaci@egi.eu), Nuno Ferreira (nuno.ferreira@egi.eu)
  • IFCA: Jesus Marco de Lucas (marco@ifca.unican.es)
  • Department of Biodiversity and Evolutionary Biology at MNCN (Museo Nacional de Ciencias Naturales, Madrid): Etienne Kornobise (ekornobis@gmail.com)
VERCE Earthquake and seismology research addresses fundamental problems in understanding the Earth's internal wave sources and properties, thereby aiding society in the management of natural hazards, energy resources, environmental changes, and national security. VERCE is supporting this effort by developing a data-intensive e-science environment to enable innovative data analysis and data modelling methods that fully exploit the increasing wealth of open data generated by the observational and monitoring systems of the global seismology community.
  • Use Case : evaluate EGI Federated Cloud capabilities for data analysis and post-processing.
FedCloudVERCE
  • EGI: Gergely Sipos (gergely.sipos@egi.eu)
  • VERCE: andre.gemuend@scai.fraunhofer.de
WeNMR The objective of WeNMR is to optimize and extend the use of the NMR and SAXS research infrastructures through the implementation of an e-infrastructure in order to provide the user community with a platform integrating and streamlining the computational approaches necessary for NMR and SAXS data analysis and structural modelling.
  • Use Case 1: using VMs prepared with Gromacs and some other software to run MD simulations for educational purpose, possibly on multi-core VMs.
  • Use Case 2: validating and improving biomolecular NMR structures using VirtualCing (http://code.google.com/p/cing/wiki/VirtualCing), a VM equipped with a complex suite of ~25 programs. The cloud usage framework is based on a pilot job mechanism making use of the ToPoS (https://grid.sara.nl/wiki/index.php/Using_the_Grid/ToPoS) tool. Therefore, such a framework would naturally allow for execution of VirtualCing tasks across multiple cloud providers. Do notice that the framework is independent on the cloud access interface: it would work also with simple grid jobs, as far as the user-defined (or VO manager defined) VirtualCing VM is available at the grid site e.g. in a SE (or in the VO software area mounted by the WNs) and the grid job is allowed to start the VM. Technical details about its current implementation are available at http://code.google.com/p/cing/w/list. A live demonstration about the deployment and use of VirtualCing on the WNoDeS testbed of the INFN-CNAF computing centre has been shown at the EGI TF 2012 held in September: http://prezi.com/hrzwekguwfkx/virtualcing-cloud-demo-on-wnodes.
FedCloudWeNMR
  • EGI: Nuno Ferreira (nuno.ferreira@egi.eu)
  • WeNRR: Marco Verlato (marco.verlato@pd.infn.it), Alexandre Bonvin (a.m.j.j.bonvin@uu.nl)
WS-PGRADE WS-PGRADE is a portal environment for the development, execution and monitoring of workflows and workflow based parameter studies on different Distributed Computing Infrastructures (DCI).
  • Use Case 1: biologists, chemists simulating molecular docking by the autodock software tool are potential users of this use case. This use case gives the ability to run a small BOINC based desktop grid infrastructure as a DCI and to submit a pre-defined application (called autodock) to this DCI through the WS-PGRADE/gUSE portal as a (predefined) workflow.
  • Use Case 2: any scientists requiring an on-demand, scalable computing infrastructure are potential users of this use case. This use case gives the ability to run a small BOINC based desktop grid infrastructure providing virtualisation support (GBAC) on the computational resource (BOINC client). The job submission interface in this scenario is the WS-PGRADE/gUSE system where compound applications (i.e. workflows) can be easily built and executed on the BOINC based desktop grid DCI. The submitted jobs of the workflow are executed on minimal linux OS used as the virtualised environment. Scalability can be improved by attaching external (non-cloud) resources to the desktop grid server.
FedCloudWSPGRADE
  • EGI: Gergely Sipos (gergely.sipos@egi.eu)
  • WS-PGRADE: Peter Kotcauer (peter.kotcauer@sztaki.mta.hu)

Ongoing Use Cases (work in progress)

Project or Application Description Further information
BioVeL The BioVel project provides workflows for the processing of data in major areas of biodiversity research: ecological niche modelling, ecosystem functioning, and taxonomy. The community wants to explore how critical assets such as the BioVeL Portal, and web-services like OpenModeller, BioSTIF and OpenRefine can be sustained in the EGI Federated Cloud.
BSIM2 BSIM2’s mission is to boost the discovery and optimisation of new drug candidates, led by efficient computational methodologies and workflows. This startup company is exploring a set of Proof's of Concept on the EGI Federated Cloud: molecular dynamics simulations, chemoinformatics applications & workflows, and 3rd party licensed software for windows/linux are the key topics. FedCloudBSIM2
Chipster Chipster is a user-friendly analysis software for high-throughput data. It contains over 300 analysis tools for next generation sequencing (NGS), microarray, proteomics and sequence data. The goal is to deploy Chipster on the Cloud Federation. FedCloudChipster
DCH-RP Digital Cultural Heritage Roadmap for Preservation (DCH-RP) coordination action aims to look at best practices for preservation standards. The use case wants to run a set of data preservation services at the EGI Federated Cloud. FedCloudDCHRP
DRHIM This community aims to predict, manage and mitigate the risks associated to extreme weather phenomena. A set of hydrological models are to be explored using EGI Cloud Federation. FedCloudDRIHM
ENVRI The ENVRI aims in developing common capabilities including software and services of the environmental and e-infrastructure communities. A set of use cases are being tackled in the Cloud Federation, namely the hosting of data access and dissemination exploring the SaaS model, and data processing via IaaS model. EGI_ENVRI






Concluded Use Cases (work in progress)

Project or Application Description Further information
Catania Science Gateway Framework The Catania Science Gateway Framework (CSGF) provides application developers with a tool to create Science Gateways. CSGF is able to manage AuthN/AuthZ and to interact with different DCIs (grid, cloud, HPC, local, etc.). The CSGF use case exploits EGI Cloud capabilities through via a SaaS service model. Furthermore, the CSGF can orchestrate the cloud services through their OCCI-compliant and rOCCI-enabled interfaces acting as infrastructure broker.
BNCWeb BNCWeb is an interface to the British National Corpus, being a key reference work in English studies, linguistics and language teaching and is widely used in a wide variety of computational linguistic applications. The use case aims to offer BNCWeb services via EGI FedCloud. FedCloudBNCweb
European Space Agency In the context of the Helix Nebula initiative, the European Space Agency organized a Proof of Concept using EGI Federated Cloud resources. The objective is to prove the interoperability between commercial (Helix Nebula) and academic (EGI Federated Cloud) cloud providers and to prove the possibility to provide processing services to scientists using the Federated Cloud IaaS system. ESA target is volcano and earthquake monitoring in the context of the SuperSites Exploitation Platform project. FedCloudESAPoC
WeNMR The objective of WeNMR is to optimize and extend the use of the NMR and SAXS research infrastructures through the implementation of an e-infrastructure in order to provide the user community with a platform integrating and streamlining the computational approaches necessary for NMR and SAXS data analysis and structural modelling. FedCloudWeNMR
Peachnote Peachnote.com is a music score search engine and analysis platform. The system is the first of its kind and can be thought as an analog of Google Books Ngram Viewer and Google Books search for music scores. Peachnote provides visitors and researchers access to a massive amount of symbolic music data. FedCloudPeachnote
Cranfield University One of the gosal of this use case is to evaluate the EGI Federated Cloud in order to organise hands-on training for students. Another topic of interest is the exploitation of the Federated Cloud capabilities for resource allocation and scheduling algorithms for federated cloud systems. FedCloudCranfield
DIRAC The DIRAC interware project provides a framework for building ready to use distributed computing systems. Regarding cloud resources, DIRAC is currently integrated with Amazon EC2, OpenNebula, OpenStack and CloudStack. DIRAC wants to explore Iaas cloud model for Monte Carlo simulations, and to explore VMDIRAC as portal for VM scheduler, with third party job broker. FedCloudDIRAC
European Space Agency In the context of the Helix Nebula initiative, the European Space Agency organized a Proof of Concept using EGI Federated Cloud resources. The objective is to prove the interoperability between commercial (Helix Nebula) and academic (EGI Federated Cloud) cloud providers and to prove the possibility to provide processing services to scientists using the Federated Cloud IaaS system. FedCloudESAPoC
FTK FTK – Research Institute for Telecommunication and Cooperation – consists of a team with broad and profound expertise in various fields such as consulting, research, information, innovation- and project management, event organisation and web development. FTK would like to evaluate the EGI Federated Cloud capabilities. FedCloudFTK