Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Training infrastructure"

From EGIWiki
Jump to navigation Jump to search
 
(27 intermediate revisions by 4 users not shown)
Line 3: Line 3:
= Introduction to the EGI training infrastructure =
= Introduction to the EGI training infrastructure =


During 2015, under the coordination of the EGI-Engage project, the EGI community established a cloud based e-infrastructure specifically for training. The training exists as a resource pool - a so called Virtual Organisation (VO) - of the EGI Federated Cloud infrastructure. The training infrastructure provides ‘Infrastructure as a Service’ clouds (IaaS), i.e. physical resources (CPU, storage) - as well as access services (login, application catalogue and application management portal) for face-to-face events, online training courses or self-paced learning modules.
During 2015, under the coordination of the EGI-Engage project, the EGI community established a cloud based e-infrastructure specifically for training. The training infrastructure exists as a resource pool - a so called Virtual Organisation (VO) - within the [Federated_Cloud_user_support|EGI Federated Cloud infrastructure]. The training infrastructure provides ‘Infrastructure as a Service’ clouds (IaaS), i.e. physical resources (CPU, storage), as well as access services (login, application catalogue and application management portal) for face-to-face events, online training courses or self-paced learning modules.
The training infrastructure is integrated with the EGI authentication-authorisation service (CheckIn) that allows trainers to generate short-living user accounts for training participants. Such accounts can identify students individually, and for a limited lifetime - typically few hours or days, depending on the length of the training event - allow them to interact with the training infrastructure sites and services.
The training infrastructure is integrated with the EGI authentication-authorisation service (CheckIn) that allows trainers to generate short-living user accounts for training participants. Such accounts can identify students individually, and for a limited lifetime - typically few hours or days, depending on the length of the training event - allow them to interact with the training infrastructure sites and services.
The infrastructure currently includes enough capacity to scale up to class-room size audiences, approximately up to 100 participants. 


Introductory slides and poster of the Training Infrastructure: https://indico.egi.eu/indico/contributionDisplay.py?contribId=122&confId=2544
Introductory slides and poster of the Training Infrastructure: https://indico.egi.eu/indico/contributionDisplay.py?contribId=122&confId=2544
(These were prepared for a Demo at the EGI Community Forum event in Bari, 2015 November. Outdated in some parts.)
(These were prepared for a Demo at the EGI Community Forum event in Bari, 2015 November. Outdated in some parts.)
'''Do you want to deploy a custom training environment on this infrastructure? Please email support@egi.eu.'''


= Usage models  =
= Usage models  =
Line 32: Line 32:
'''Available clouds:'''  
'''Available clouds:'''  


*CESNET OpenNebula cloud site (Czech Republic)  
*CESNET-MCC - OpenStack cloud site (Czech Republic)  
**64 vCPUs, 110 GB of RAM, 1 TB of persistent storage  
**64 vCPUs, 110 GB of RAM, 1 TB of persistent storage  
**Contact: Miroslav Ruda <ruda@ics.muni.cz>, Boris Parak <parak@cesnet.cz>
**Contact: Miroslav Ruda <ruda@ics.muni.cz>, Boris Parak <parak@cesnet.cz>


*INFN-CATANIA-STACK OpenStack cloud site (Italy)  
*INFN-CATANIA-STACK - OpenStack cloud site (Italy)  
**60 vCPUs, 100 GB of RAM, 1TB of storage volume  
**20 vCPUs, 50 GB of RAM, 1TB of storage volume  
**Contact: roberto.barbera@ct.infn.it, giuseppe.platania@ct.infn.it, sg-licence@ct.infn.it
**Contact: roberto.barbera@ct.infn.it, giuseppe.platania@ct.infn.it, sg-licence@ct.infn.it


'''Clouds under maintenance:'''
*IISAS-GPUCloud - OpenStack cloud site (Slovakia)
**12 vCPU cores, 2 GPU and 48GB of RAM
*** In case of special request, these resources can be doubled but must be reserved in advance
**Contact: viet.ui@savba.sk, cloud-admin@savba.sk


*UKIM OpenNebula cloud site (Macedonia)
*IISAS-FedCloud - OpenStack cloud site (Slovakia)
**48 vCPUs, 48 GB of RAM, 48 public IP addresses
**16 vCPU cores with 1.6GB of RAM per core
**Contact: Boro Jakimovski <boro.jakimovski@finki.ukim.mk>
 
*CETA-CIEMAT OpenStack cloud site (Spain)
**20 physical CPU cores (or virtual cores), 40 GB of RAM, 5.4TB storage shared among all the VOs (CDMI is not provided), 20 public IPs
**Contact: miguelangel.diaz@externos.ciemat.es, admin.iaas@ceta-ciemat.es
 
''' Clouds under finalization:'''
*IISAS-GPU OpenStack cloud site (Slovakia)
**12 vCPU cores, 2 GPU and 48GB RAM of storage volume
*** In case of special request, these resources can be doubled but must be reserved in advance
**Contact: viet.ui@savba.sk, cloud-admin@savba.sk
**Contact: viet.ui@savba.sk, cloud-admin@savba.sk


* HG-09-Okeanos-Cloud Synnepho (Greece)
* IFCA-LCG2 OpenStack (Spain)
**Contact: kkoum@admin.grnet.gr
**20 vCPU cores, 50GB of RAM in total, 1TB of block storage, 1 public IP address
**Additional IP addresses can be provided, upon request, for the specific training event
**Contact: aloga@ifca.unican.es


'''Cloud not anymore available:'''
'''Cloud not anymore available:'''
Line 98: Line 93:
* Cloud Tutorial at EUDAT summer School, Heraklion, Crete (03-07/07/2017). Training material from the event: https://www.eudat.eu/eudat-summer-school
* Cloud Tutorial at EUDAT summer School, Heraklion, Crete (03-07/07/2017). Training material from the event: https://www.eudat.eu/eudat-summer-school
* CODATA-RDA Research Data Science Summer School, Trieste (21 July 2017). Training materials from the event: https://documents.egi.eu/document/3168
* CODATA-RDA Research Data Science Summer School, Trieste (21 July 2017). Training materials from the event: https://documents.egi.eu/document/3168
* Scipion tutorial on Cloud. Training materials from the event: http://i2pc.es/instruct-course-on-image-processing-for-electron-microscopy-in-the-cloud-madrid-january-17-19-2018/
* International Summer School “Data Management in Environmental & Earth Science Infrastructures: Theory & Practice”, Lecce (09-13 July 2019). Training info and material from the event: https://documents.egi.eu/public/ShowDocument?docid=3341
* CODATA-RDA Research Data Science Summer School, Trieste (17 Aug 2018). Training materials from the event: https://documents.egi.eu/public/ShowDocument?docid=3349
* NGSchool 2018. Training info and materials from the event: https://ngschool.eu/
* 3rd Int'l Summer School on Data Science (SSDS 2018). Training info and materials from the event: https://sites.google.com/site/ssdatascience2018/
* Training for PhD students at the University of Genoa (04 June 2019). Training info and materials from the event: http://dottorato.dicca.unige.it/eng/schede_corsi_2019/eScience%20new%20information%20technologies%20for%20research.pdf
* Introduction to Jupyter and Open Science - Training, Yerevan (27 September 2019). Training materials from the event: http://go.egi.eu/eapec2019
* HPC graduate class (August - November 2019) at UNICAM (Brazil), 15 meetings
* NGSChool 2019 (October, 24-31). Agenda page: https://ngschool.eu/ngschool2019/
* Open Science with Jupyter, Zenodo and Binder (tutorial), 04 Dec. 2019. Training info and materials from the event: http://go.egi.eu/bigdata2019
* Hercules European School (April 2020). Agenda page: http://hercules-school.eu/
= Advanced Usage =
To configure an HTC clusters on top of IaaS infastructure, please check the following
[[Deploy HTC clusters on top of IaaS infrastructure | Deploy HTC clusters on top of IaaS infrastructure ]] guide.


= Booking the infrastructure =
= Booking the infrastructure =


Do you want to book the infrastructure for a course? Please send a request through https://www.egi.eu/services/training-infrastructure/
The infrastructure currently includes enough capacity to scale up to class-room size audiences, approximately up to 100 participants. Do you want to book the infrastructure for a course? Please send a request through https://www.egi.eu/services/training-infrastructure/


[[Category:Community_Engagement]]
[[Category:Community_Engagement]]

Latest revision as of 07:36, 8 July 2020

Engagement overview Community requirements Community events Training EGI Webinars Documentations



Introduction to the EGI training infrastructure

During 2015, under the coordination of the EGI-Engage project, the EGI community established a cloud based e-infrastructure specifically for training. The training infrastructure exists as a resource pool - a so called Virtual Organisation (VO) - within the [Federated_Cloud_user_support|EGI Federated Cloud infrastructure]. The training infrastructure provides ‘Infrastructure as a Service’ clouds (IaaS), i.e. physical resources (CPU, storage), as well as access services (login, application catalogue and application management portal) for face-to-face events, online training courses or self-paced learning modules. The training infrastructure is integrated with the EGI authentication-authorisation service (CheckIn) that allows trainers to generate short-living user accounts for training participants. Such accounts can identify students individually, and for a limited lifetime - typically few hours or days, depending on the length of the training event - allow them to interact with the training infrastructure sites and services.

The infrastructure currently includes enough capacity to scale up to class-room size audiences, approximately up to 100 participants.

Introductory slides and poster of the Training Infrastructure: https://indico.egi.eu/indico/contributionDisplay.py?contribId=122&confId=2544 (These were prepared for a Demo at the EGI Community Forum event in Bari, 2015 November. Outdated in some parts.)

Usage models

The training infrastructure is suitable for two types of courses:

  1. Cloud computing courses: Such courses teach students about IaaS clouds and the way Virtual Machine images, storage blocks and other types of ‘low level’ resources are managed. For such courses In this operational mode the infrastructure the trainer does not have to deploy applications or online services on the infrastructure in advance of the course. The applications/services will be deployed by the students themselves as training exercises. Such courses typically target programmers or other rather technical members of scientific communities or projects.
  2. Scientific courses: Such courses teach scientists or programmers about a specific software that’s relevant for their work. For example a specific gene sequence analysis software suite, an earthquake visualisation tool, a data processing pipeline. In this operational mode the the trainer deploys the domain specific application/tool on the training infrastructure before the training and the students interact directly with those applications/tools without even knowing where those are deployed and running. Depending on how computationally or data heavy the student exercises are, multiple students can share a single software deployment, or each student can have his/her own dedicated deployment. The configuration can be controlled by the teacher when the setup is deployed. have dedicated training environments, and the community can benefit from the easy deployment, predictability and repeatability of courses.

In both operational case the deployment of applications/tools/services can happen in the form of ‘Virtual Machine images’ (VMIs), and block storages - the latter basically behaving like a virtual USB drive that can be attached/detached to VMIs to provide data and storage space for applications.

VMIs can be deployed on the infrastructure through the EGI Applications Marketplace (https://appdb.egi.eu/browse/cloud), a growing catalogue that includes both basic applications (e.g. latest version of clean Linux deployments) and more specialised applications (e.g. Jupyter Notebook). A specific section within the catalogue is reserved for training-related application VMIs (https://appdb.egi.eu/store/vo/training.egi.eu).

The Applications Marketplace also includes a VMI management dashboard (https://dashboard.appdb.egi.eu), offering a Web interface for both trainers and students to deploy and manage VMIs from the Marketplace on the training infrastructure.

Available resources

The infrastructure currently federates the following resources:

Sites through the training.egi.eu Virtual Organisation (VO):

Available clouds:

  • CESNET-MCC - OpenStack cloud site (Czech Republic)
    • 64 vCPUs, 110 GB of RAM, 1 TB of persistent storage
    • Contact: Miroslav Ruda <ruda@ics.muni.cz>, Boris Parak <parak@cesnet.cz>
  • INFN-CATANIA-STACK - OpenStack cloud site (Italy)
    • 20 vCPUs, 50 GB of RAM, 1TB of storage volume
    • Contact: roberto.barbera@ct.infn.it, giuseppe.platania@ct.infn.it, sg-licence@ct.infn.it
  • IISAS-GPUCloud - OpenStack cloud site (Slovakia)
    • 12 vCPU cores, 2 GPU and 48GB of RAM
      • In case of special request, these resources can be doubled but must be reserved in advance
    • Contact: viet.ui@savba.sk, cloud-admin@savba.sk
  • IISAS-FedCloud - OpenStack cloud site (Slovakia)
    • 16 vCPU cores with 1.6GB of RAM per core
    • Contact: viet.ui@savba.sk, cloud-admin@savba.sk
  • IFCA-LCG2 OpenStack (Spain)
    • 20 vCPU cores, 50GB of RAM in total, 1TB of block storage, 1 public IP address
    • Additional IP addresses can be provided, upon request, for the specific training event
    • Contact: aloga@ifca.unican.es

Cloud not anymore available:

  • BIFI OpenStack cloud site (Spain)
    • 100 vCPUs, 200 GB of RAM, 1TB for storage volumes, 50 public IP addresses
    • Contact: Ruben Valles <rvalles@bifi.es>

Documentation:


Do you want to join as a resource provider? Please email support@egi.eu.

Modules and examples

The training infrastructure was used in July 2015 for two face-to-face training courses that trained programmers about the use of the EGI Federated Cloud (so the first operational model above). The focus of these courses were on the use of the rOCCI client interface to interact with the EGI cloud (instantiate, delete, access VMs). During these courses the training infrastructure included three cloud sites (CESNET, BIFI, UKIM) and hosted approx 25 student per event. Before the event the trainers prepared a basic VM that included the rOCCI client (with the training VO pre-configured in it), one user account for each student with a short-term certificate copied under each (with 24h lifetime). One instance of the VM was started on the infrastructure by the trainers before the event and the student were provided with login details for the user accounts. After logging in the students could use the OCCI commands of the rOCCI client with their own short-term certificates and could manage virtual machines on the 3 cloud sites of the training infrastructure. The presentations and hands-on materials from the teo events are available here:

Advanced Usage

To configure an HTC clusters on top of IaaS infastructure, please check the following Deploy HTC clusters on top of IaaS infrastructure guide.

Booking the infrastructure

The infrastructure currently includes enough capacity to scale up to class-room size audiences, approximately up to 100 participants. Do you want to book the infrastructure for a course? Please send a request through https://www.egi.eu/services/training-infrastructure/