View source for GPGPU-FedCloud

{{Template:EGI-Engage menubar}} {{TOC_right}} 

= Objective =

To provide support for accelerated computing in EGI-Engage federated cloud.


= Participants =

Viet Tran (IISAS)

Jan Astalos (IISAS)

Miroslav Dobrucky (IISAS)

= Current status =

A working site with GPGPU in EGI federated cloud https://cloudmon.egi.eu/nagios/cgi-bin/status.cgi?host=nova3.ui.savba.sk

HW configuration: 
 IBM dx360 M4 server with two NVIDIA Tesla K20 accelerators.
 Ubuntu 14.04.2 LTS with KVM/QEMU, PCI passthrough virtualization of GPU cards.

SW configuration:
 Base OS: Ubuntu 14.04.2 LTS
 Hypervisor: KVM
 Middleware: Openstack Kilo

EGI federated cloud configuration:
 GOCDB: IISAS-GPUCloud, https://goc.egi.eu/portal/index.php?Page_Type=Site&id=1485
 Openstack endpoint: https://keystone3.ui.savba.sk:5000/v2.0
 OCCI endpoint: https://nova3.ui.savba.sk:8787
 Supported VOs: fedcloud.egi.eu, ops, dteam, moldyngrid

= How to use GPGPU on IISAS-GPUCloud =
For EGI users:
 Join EGI federated cloud https://wiki.egi.eu/wiki/Federated_Cloud_user_support#Quick_Start
 
 Get VOMS proxy certificate from fedcloud.egi.eu or any supported VO (voms-proxy-init --voms fedcloud.egi.eu -rfc)
  
 Choose suitable gpu.* flavor (e.g. gpu.medium or gpu.large) (OCCI users: resource_tpl#6 or resource_tpl#7)
 
 Choose suitable image (e.g. Ubuntu-14.4, OCCI users: os_tpl#3e1ad5a5-4d5a-4dbb-8b93-5c329129d3e6)
  
 Create a VM with the selected image and flavor (OCCI users: occi  --endpoint  https://nova3.ui.savba.sk:8787/ \
                   --auth x509 --user-cred $X509_USER_PROXY --voms --action create --resource compute \
                   --mixin os_tpl#3e1ad5a5-4d5a-4dbb-8b93-5c329129d3e6 --mixin resource_tpl#6 \
                   --attribute occi.core.title="Testing GPU")
 
 Log in the VM and use it as your own GPU server.

For access to IISAS-GPUCloud via portal:
 Contact with cloud_admin _at_ savba.sk to get account and access to full-featured graphical portal

= Progress =

* May 2015
** Review of available technologies
** GPGPU virtualisation in KVM/QEMU
** Performance testing of passthrough

 HW configuration: 
 IBM dx360 M4 server with two NVIDIA Tesla K20 accelerators.
 Ubuntu 14.04.2 LTS with KVM/QEMU, PCI passthrough virtualization of GPU cards.

 Tested application:
 NAMD molecular dynamics simulation (CUDA version), STMV test example (http://www.ks.uiuc.edu/Research/namd/).

 Performance results:
 Tested application runs 2-3% slower in virtual machine compared to direct run on tested server.
 If hyperthreading is enabled on compute server, vCPUs have to be pinned to real cores so that
 whole cores will be dedicated to one VM. To avoid potential performance problems, hyperthreading 
 should be switched off.
 
* June 2015
** Creating cloud site with GPGPU support
 Configuration: master node, 2 worker nodes (IBM dx360 M4 servers, see above)
 Base OS: Ubuntu 14.04.2 LTS
 Hypervisor: KVM
 Middleware: Openstack Kilo

* July 2015
** Creating cloud site with GPGPU support
 Cloud site created at keystone3.ui.savba.sk, master + two worker nodes, configuration reported above
 Creating VM images for GPGPU (based on Ubuntu 14.04, GPU driver and libraries)

* August 2015
** Testing cloud site with GPGPU support
 Performance testing and tuning with GPGPU in Openstack 
  - comparing performance of cloud-based VM with non-cloud virtualization and physical machine, finding discrepancies and tuning them
  - setting CPU flavor in Openstack nova (performance optimization) 
  - Adjusting Openstack scheduler

 Starting process of integration of the site to EGI FedCloud
  - Keystone VOMS support being integrated
  - OCCI in preparation, installation planned in September

* September 2015
  Continue integration to EGI-FedCloud

* Next steps
  Full integration, certification and production support

= [https://wiki.egi.eu/wiki/EGI-Engage:TASK_JRA2.4_Accelerated_Computing Back to Accelerated Computing task] =