Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "FedCloudJAMSPoC"

From EGIWiki
Jump to navigation Jump to search
(Created page with "Category: Technology Category: Fedcloud-tf {{FedCloudUseCases | FCUC_Status = active| FCUC_StartDate = 29/05/2014 | FCUC_EndDate = -| FCUC_EGIName = Diego S...")
 
 
(11 intermediate revisions by one other user not shown)
Line 1: Line 1:
[[Category: Technology ]]
{{Fedcloud_Menu}}
[[Category: Fedcloud-tf]]
 
{{FCloudCom_menubar}}
 
{{TOC_right}}
{{FedCloudUseCases |  
{{FedCloudUseCases |  
FCUC_Status    = active|  
FCUC_Status    = Closed|  
FCUC_StartDate = 29/05/2014 |  
FCUC_StartDate = 27/05/2014 |  
FCUC_EndDate  = -|   
FCUC_EndDate  = 18/09/2014|   
FCUC_EGIName  =  Diego Scardaci / diego.scardaci@egi.eu  |  
FCUC_EGIName  =  Diego Scardaci / diego.scardaci@egi.eu  |  
FCUC_EXTName  = Christian Fischer / christian.fischer.2@uni-jena.de |  
FCUC_EXTName  = Christian Fischer / christian.fischer.2@uni-jena.de |  
Line 11: Line 14:
JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams).
JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams).


JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week.  
JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week.
Aim of this proof-of-concept was the evaluation of the EGI Federated Cloud as alternative to the community computing resources to execute JAMS computation. The community already owns enough computing resources for their current needs but its workload should increase in a near future.|
FCUC_Tasks =
Run JAMS on the EGI Federated Cloud


http://jams.uni-jena.de/ |
'''Requirements:'''
* 1 VM with linux operating system and a (oracle) java runtime environment
* more than 16GB memory
* 8 or more computing cores (since parallelization is thread-based)


FCUC_Tasks =
==Outcome==
=====Use case =====
The community was able to execute standard JAMS simulation in the EGI Federated Cloud without major issue. They used the [https://appdb.egi.eu/store/site/prisma-infn-bari PRISMA-INFN-BARI] site for their tests.
Run on the EGI Federated Cloud a system to support modeling of physiological systems in the phase of calibrating model parameters and in the phase of simulating different scenarios. The loosely coupled part of the system is deployed in a remote distributed computational capacity. The system capabilities is accessible via a web application and allows user to focus on experimental data, names of parameters, visual control of the calibration computation and hide unnecessary complexity of the remote subsystems computation. The data of real experiments and simulations are stored and provided for further research.
 
Two minor issues were identified:
* slow internet connection to the server deployed on the cloud site;
* VMs seam to increase the computation time. They tested a standard simulation, which runs in about 10 seconds on their local machine with an i7-3520M@2.9Ghz and in about 15 seconds on their HPC Server with AMD Opterons 6172@2.1Ghz. On the EGI Federated Cloud it required more than 30 seconds (without any parallelization). Anyway, the sequential computation time is not a problem since they run the model in parallel.
 
The community will ask the EGI Federated Cloud for resources when their local resources will be not enough for their computations anymore.


'''Requirements:'''
The community is willing to pay for resources and they are interested to know the EGI pricelist in order to take into account these costs in future project proposals.
* 1 VM
* Number of CPU cores: at least 10 CPU per virtual machine - (12 CPU)
* RAM requirements per CPU core: at least 300 MB per core - (4 GB)
* Storage requirements (non persistent): 1 GB per core free, (12 GB free) - (24GB non persistent disc with operating system)
* Inbound/Outbound during execution - 10kB-100kB per sec
* Licensed software - MS Windows 2008 R2 Datacenter (the developer has an MSDN licence limited for development and testing purpose)
* Operating System - MS Windows 2008 R2 Datacenter: VM image already available, a remote desktop system has to be added.
* ports - 48048-48060 for task/results exchange for incoming communication, 80,443, for HTTP outgoing synchronization with web services (physiome.lf1.cuni.cz)
* public ip addresses
|  
|  


FCUC_Files =  
FCUC_Files =  
* Phisiome project, slide presented at the EGI CF 2014: https://indico.egi.eu/indico/contributionDisplay.py?sessionId=10&contribId=51&confId=1994 |
* JAMS web site: http://jams.uni-jena.de/
* JAMS documentation: http://jams.uni-jena.de/5582.0.html?&L=2 |
}}
}}

Latest revision as of 16:33, 7 May 2015

Overview For users For resource providers Infrastructure status Site-specific configuration Architecture



Federated Cloud Communities menu: Home • Production use cases • Under development use cases • Closed use cases • High level tools use cases



General Information

  • Status: Closed
  • Start Date: 27/05/2014
  • End Date: 18/09/2014
  • EGI.eu contact: Diego Scardaci / diego.scardaci@egi.eu
  • External contact: Christian Fischer / christian.fischer.2@uni-jena.de

Short Description

JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams).

JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week. Aim of this proof-of-concept was the evaluation of the EGI Federated Cloud as alternative to the community computing resources to execute JAMS computation. The community already owns enough computing resources for their current needs but its workload should increase in a near future.

Use Case

Run JAMS on the EGI Federated Cloud

Requirements:

  • 1 VM with linux operating system and a (oracle) java runtime environment
  • more than 16GB memory
  • 8 or more computing cores (since parallelization is thread-based)

Outcome

The community was able to execute standard JAMS simulation in the EGI Federated Cloud without major issue. They used the PRISMA-INFN-BARI site for their tests.

Two minor issues were identified:

  • slow internet connection to the server deployed on the cloud site;
  • VMs seam to increase the computation time. They tested a standard simulation, which runs in about 10 seconds on their local machine with an i7-3520M@2.9Ghz and in about 15 seconds on their HPC Server with AMD Opterons 6172@2.1Ghz. On the EGI Federated Cloud it required more than 30 seconds (without any parallelization). Anyway, the sequential computation time is not a problem since they run the model in parallel.

The community will ask the EGI Federated Cloud for resources when their local resources will be not enough for their computations anymore.

The community is willing to pay for resources and they are interested to know the EGI pricelist in order to take into account these costs in future project proposals.

Additional Files

Retrieved from "https://wiki.egi.eu/w/index.php?title=FedCloudJAMSPoC&oldid=79327"