Difference between revisions of "FedCloudJAMSPoC"
(10 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
{{Fedcloud_Menu}} | |||
{{FCloudCom_menubar}} | |||
{{TOC_right}} | |||
{{FedCloudUseCases | | {{FedCloudUseCases | | ||
FCUC_Status = | FCUC_Status = Closed| | ||
FCUC_StartDate = | FCUC_StartDate = 27/05/2014 | | ||
FCUC_EndDate = | FCUC_EndDate = 18/09/2014| | ||
FCUC_EGIName = Diego Scardaci / diego.scardaci@egi.eu | | FCUC_EGIName = Diego Scardaci / diego.scardaci@egi.eu | | ||
FCUC_EXTName = Christian Fischer / christian.fischer.2@uni-jena.de | | FCUC_EXTName = Christian Fischer / christian.fischer.2@uni-jena.de | | ||
Line 11: | Line 14: | ||
JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams). | JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams). | ||
JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week. | JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week. | ||
Aim of this proof-of-concept was the evaluation of the EGI Federated Cloud as alternative to the community computing resources to execute JAMS computation. The community already owns enough computing resources for their current needs but its workload should increase in a near future.| | |||
FCUC_Tasks = | FCUC_Tasks = | ||
Run JAMS on the EGI Federated Cloud | Run JAMS on the EGI Federated Cloud | ||
'''Requirements:''' | '''Requirements:''' | ||
* 1 VM with linux operating system and a (oracle) java runtime environment | * 1 VM with linux operating system and a (oracle) java runtime environment | ||
* more than | * more than 16GB memory | ||
* | * 8 or more computing cores (since parallelization is thread-based) | ||
==Outcome== | |||
The community was able to execute standard JAMS simulation in the EGI Federated Cloud without major issue. They used the [https://appdb.egi.eu/store/site/prisma-infn-bari PRISMA-INFN-BARI] site for their tests. | |||
Two minor issues were identified: | |||
* slow internet connection to the server deployed on the cloud site; | |||
* VMs seam to increase the computation time. They tested a standard simulation, which runs in about 10 seconds on their local machine with an i7-3520M@2.9Ghz and in about 15 seconds on their HPC Server with AMD Opterons 6172@2.1Ghz. On the EGI Federated Cloud it required more than 30 seconds (without any parallelization). Anyway, the sequential computation time is not a problem since they run the model in parallel. | |||
The community will ask the EGI Federated Cloud for resources when their local resources will be not enough for their computations anymore. | |||
The community is willing to pay for resources and they are interested to know the EGI pricelist in order to take into account these costs in future project proposals. | |||
| | | | ||
FCUC_Files = | FCUC_Files = | ||
* JAMS web site: http://jams.uni-jena.de/ | |||
* JAMS documentation: http://jams.uni-jena.de/5582.0.html?&L=2 | | * JAMS documentation: http://jams.uni-jena.de/5582.0.html?&L=2 | | ||
}} | }} |
Latest revision as of 16:33, 7 May 2015
Overview | For users | For resource providers | Infrastructure status | Site-specific configuration | Architecture |
Federated Cloud Communities menu: | Home • | Production use cases • | Under development use cases • | Closed use cases • | High level tools use cases |
General Information
- Status: Closed
- Start Date: 27/05/2014
- End Date: 18/09/2014
- EGI.eu contact: Diego Scardaci / diego.scardaci@egi.eu
- External contact: Christian Fischer / christian.fischer.2@uni-jena.de
Short Description
JAMS is an java-based, open-source software platform that has been especially designed to address the demands of a process-based hydrological model development and various aspects of model application (https://appdb.egi.eu/store/software/jams).
JAMS is a framework to build up complex models out of simple components. Several hydrological models were implemented within JAMS (e.g. J2000, J2000g). Usually those models are applied to simulate hydrological dynamics in catchments with a size of 1km² to 100,000 km² in a temporal time step of hours to months. Typically, the computing time of a single model run is in the range from minutes to hours (on a single workstation). However, those models have many parameters, which must be estimated indirectly during a calibration process. For model calibration an evolutionary optimization algorithms was adopted, which evaulate the model several thousand times. Therefore this calibration process takes days to weeks on a single workstation. To speed up this process, we implemented a thread-based parallelization in JAMS. This thread-based parallelization is used for example to evaluated several parameter combinations in parallel during optimization. The model calibration is carried out frequently. Sometimes more than 10 times a week. Aim of this proof-of-concept was the evaluation of the EGI Federated Cloud as alternative to the community computing resources to execute JAMS computation. The community already owns enough computing resources for their current needs but its workload should increase in a near future.
Use Case
Run JAMS on the EGI Federated Cloud
Requirements:
- 1 VM with linux operating system and a (oracle) java runtime environment
- more than 16GB memory
- 8 or more computing cores (since parallelization is thread-based)
Outcome
The community was able to execute standard JAMS simulation in the EGI Federated Cloud without major issue. They used the PRISMA-INFN-BARI site for their tests.
Two minor issues were identified:
- slow internet connection to the server deployed on the cloud site;
- VMs seam to increase the computation time. They tested a standard simulation, which runs in about 10 seconds on their local machine with an i7-3520M@2.9Ghz and in about 15 seconds on their HPC Server with AMD Opterons 6172@2.1Ghz. On the EGI Federated Cloud it required more than 30 seconds (without any parallelization). Anyway, the sequential computation time is not a problem since they run the model in parallel.
The community will ask the EGI Federated Cloud for resources when their local resources will be not enough for their computations anymore.
The community is willing to pay for resources and they are interested to know the EGI pricelist in order to take into account these costs in future project proposals.
Additional Files
- JAMS web site: http://jams.uni-jena.de/
- JAMS documentation: http://jams.uni-jena.de/5582.0.html?&L=2