|Engagement overview||Community requirements||Community events||Training||EGI Webinars||Documentations|
Introduction to the EGI training infrastructure
During 2015, under the coordination of the EGI-Engage project, the EGI community established a cloud based e-infrastructure specifically for training. The training infrastructure exists as a resource pool - a so called Virtual Organisation (VO) - within the [Federated_Cloud_user_support|EGI Federated Cloud infrastructure]. The training infrastructure provides ‘Infrastructure as a Service’ clouds (IaaS), i.e. physical resources (CPU, storage), as well as access services (login, application catalogue and application management portal) for face-to-face events, online training courses or self-paced learning modules. The training infrastructure is integrated with the EGI authentication-authorisation service (CheckIn) that allows trainers to generate short-living user accounts for training participants. Such accounts can identify students individually, and for a limited lifetime - typically few hours or days, depending on the length of the training event - allow them to interact with the training infrastructure sites and services.
The infrastructure currently includes enough capacity to scale up to class-room size audiences, approximately up to 100 participants.
Introductory slides and poster of the Training Infrastructure: https://indico.egi.eu/indico/contributionDisplay.py?contribId=122&confId=2544 (These were prepared for a Demo at the EGI Community Forum event in Bari, 2015 November. Outdated in some parts.)
The training infrastructure is suitable for two types of courses:
- Cloud computing courses: Such courses teach students about IaaS clouds and the way Virtual Machine images, storage blocks and other types of ‘low level’ resources are managed. For such courses In this operational mode the infrastructure the trainer does not have to deploy applications or online services on the infrastructure in advance of the course. The applications/services will be deployed by the students themselves as training exercises. Such courses typically target programmers or other rather technical members of scientific communities or projects.
- Scientific courses: Such courses teach scientists or programmers about a specific software that’s relevant for their work. For example a specific gene sequence analysis software suite, an earthquake visualisation tool, a data processing pipeline. In this operational mode the the trainer deploys the domain specific application/tool on the training infrastructure before the training and the students interact directly with those applications/tools without even knowing where those are deployed and running. Depending on how computationally or data heavy the student exercises are, multiple students can share a single software deployment, or each student can have his/her own dedicated deployment. The configuration can be controlled by the teacher when the setup is deployed. have dedicated training environments, and the community can benefit from the easy deployment, predictability and repeatability of courses.
In both operational case the deployment of applications/tools/services can happen in the form of ‘Virtual Machine images’ (VMIs), and block storages - the latter basically behaving like a virtual USB drive that can be attached/detached to VMIs to provide data and storage space for applications.
VMIs can be deployed on the infrastructure through the EGI Applications Marketplace (https://appdb.egi.eu/browse/cloud), a growing catalogue that includes both basic applications (e.g. latest version of clean Linux deployments) and more specialised applications (e.g. Jupyter Notebook). A specific section within the catalogue is reserved for training-related application VMIs (https://appdb.egi.eu/store/vo/training.egi.eu).
The Applications Marketplace also includes a VMI management dashboard (https://dashboard.appdb.egi.eu), offering a Web interface for both trainers and students to deploy and manage VMIs from the Marketplace on the training infrastructure.
The infrastructure currently federates the following resources:
Sites through the training.egi.eu Virtual Organisation (VO):
- CESNET-MCC - OpenStack cloud site (Czech Republic)
- 64 vCPUs, 110 GB of RAM, 1 TB of persistent storage
- Contact: Miroslav Ruda <email@example.com>, Boris Parak <firstname.lastname@example.org>
- INFN-CATANIA-STACK - OpenStack cloud site (Italy)
- 20 vCPUs, 50 GB of RAM, 1TB of storage volume
- Contact: email@example.com, firstname.lastname@example.org, email@example.com
- IISAS-GPUCloud - OpenStack cloud site (Slovakia)
- 12 vCPU cores, 2 GPU and 48GB of RAM
- In case of special request, these resources can be doubled but must be reserved in advance
- Contact: firstname.lastname@example.org, email@example.com
- 12 vCPU cores, 2 GPU and 48GB of RAM
- IISAS-FedCloud - OpenStack cloud site (Slovakia)
- 16 vCPU cores with 1.6GB of RAM per core
- Contact: firstname.lastname@example.org, email@example.com
- IFCA-LCG2 OpenStack (Spain)
- 20 vCPU cores, 50GB of RAM in total, 1TB of block storage, 1 public IP address
- Additional IP addresses can be provided, upon request, for the specific training event
- Contact: firstname.lastname@example.org
Cloud not anymore available:
- BIFI OpenStack cloud site (Spain)
- 100 vCPUs, 200 GB of RAM, 1TB for storage volumes, 50 public IP addresses
- Contact: Ruben Valles <email@example.com>
- User manual (for trainers): The training infrastructure is one of the Virtual Organisations of the EGI Federated cloud. The Federated Cloud user guide is applicable for usage: https://wiki.egi.eu/wiki/Federated_Cloud_user_support
- Operation Level Agreements with the sites
- VO sites and endorsed Virtual Machine images in AppDB: https://appdb.egi.eu/store/vo/training.egi.eu
- VO ID card in the EGI Operational Portal: http://operations-portal.egi.eu/vo/view/voname/training.egi.eu
Do you want to join as a resource provider? Please email firstname.lastname@example.org.
Modules and examples
The training infrastructure was used in July 2015 for two face-to-face training courses that trained programmers about the use of the EGI Federated Cloud (so the first operational model above). The focus of these courses were on the use of the rOCCI client interface to interact with the EGI cloud (instantiate, delete, access VMs). During these courses the training infrastructure included three cloud sites (CESNET, BIFI, UKIM) and hosted approx 25 student per event. Before the event the trainers prepared a basic VM that included the rOCCI client (with the training VO pre-configured in it), one user account for each student with a short-term certificate copied under each (with 24h lifetime). One instance of the VM was started on the infrastructure by the trainers before the event and the student were provided with login details for the user accounts. After logging in the students could use the OCCI commands of the rOCCI client with their own short-term certificates and could manage virtual machines on the 3 cloud sites of the training infrastructure. The presentations and hands-on materials from the teo events are available here:
- EGI Federated Cloud tutorial package (Software Carpentry Bootcamp, 2015. July 17, Feltham, London, UK). Training materials from the event: https://documents.egi.eu/document/2551
- EGI Federated Cloud tutorial package (HPCS 2015 conference, 2015. July 20-24, Amsterdam, NL). Training materials from the event: https://documents.egi.eu/document/2553
- EGI Federated Cloud for users (Training for MTA SZTAKI, 2015. October 14, Budapest, HU). Training materials from the event: https://documents.egi.eu/document/2622
- Next Generation Sequencing Analysis Training Workshop (2015. October 21, Thessaloniki, GR). Training materials from the event: https://documents.egi.eu/document/2641
- Tutorials at the EGI Community Forum in Bari, Italy (10-12 November 2015): Training materials from these: https://indico.egi.eu/indico/internalPage.py?pageId=7&confId=2544
- Introduction to the EGI Federated Cloud – the user perspective
- Dos and Don'ts for Virtual Appliance Preparation
- DIRAC Service tutorial
- Running Chipster data analysis platform in EGI Federated Cloud
- NGS Data Analysis Training Workshop
- Programming Distributed Computing Platforms with COMPSs
- A Tutorial on Hybrid Data Infrastructures: D4Science as a case study
- EGI Technical Support for ENVRI+ Use Cases Workshop, Zandvoord, The Netherlands (May 2016).
- Running CHIPSTER, Galaxy, Jupyter Notebook on the EGI Federated Cloud. (Was delivered at ELIXIR-FI workshop): https://documents.egi.eu/document/2822
- EGI Federated Cloud for developers, Krakow, Poland (28 September 2016). Training materials from the event: https://www.digitalinfrastructures.eu/content/egi-federated-cloud-developers
- UberCloud - EGI webinar: Cloud for SMEs in CAE – OpenFOAM demo: Info: https://www.egi.eu/blog/egi-and-ubercloud-webinar-for-smes-cae-openfoam-demo-20oct2016/; Webinar recording: https://www.youtube.com/watch?v=DU6LghOtrFs&feature=youtu.be
- EGI training during ENVRIplus week in Prague (14-18 November 2016). Training materials from the event: https://documents.egi.eu/document/3005
- MEDGENET-Workshop INAB, Thessaloniki, Greece (15 December 2016). Training materials from the event: http://tinyurl.com/medgenet
- Cloud Tutorial at EUDAT summer School, Heraklion, Crete (03-07/07/2017). Training material from the event: https://www.eudat.eu/eudat-summer-school
- CODATA-RDA Research Data Science Summer School, Trieste (21 July 2017). Training materials from the event: https://documents.egi.eu/document/3168
- Scipion tutorial on Cloud. Training materials from the event: http://i2pc.es/instruct-course-on-image-processing-for-electron-microscopy-in-the-cloud-madrid-january-17-19-2018/
- International Summer School “Data Management in Environmental & Earth Science Infrastructures: Theory & Practice”, Lecce (09-13 July 2019). Training info and material from the event: https://documents.egi.eu/public/ShowDocument?docid=3341
- CODATA-RDA Research Data Science Summer School, Trieste (17 Aug 2018). Training materials from the event: https://documents.egi.eu/public/ShowDocument?docid=3349
- NGSchool 2018. Training info and materials from the event: https://ngschool.eu/
- 3rd Int'l Summer School on Data Science (SSDS 2018). Training info and materials from the event: https://sites.google.com/site/ssdatascience2018/
- Training for PhD students at the University of Genoa (04 June 2019). Training info and materials from the event: http://dottorato.dicca.unige.it/eng/schede_corsi_2019/eScience%20new%20information%20technologies%20for%20research.pdf
- Introduction to Jupyter and Open Science - Training, Yerevan (27 September 2019). Training materials from the event: http://go.egi.eu/eapec2019
- HPC graduate class (August - November 2019) at UNICAM (Brazil), 15 meetings
- NGSChool 2019 (October, 24-31). Agenda page: https://ngschool.eu/ngschool2019/
- Open Science with Jupyter, Zenodo and Binder (tutorial), 04 Dec. 2019. Training info and materials from the event: http://go.egi.eu/bigdata2019
- Hercules European School (April 2020). Agenda page: http://hercules-school.eu/
To configure an HTC clusters on top of IaaS infastructure, please check the following Deploy HTC clusters on top of IaaS infrastructure guide.
Booking the infrastructure
The infrastructure currently includes enough capacity to scale up to class-room size audiences, approximately up to 100 participants. Do you want to book the infrastructure for a course? Please send a request through https://www.egi.eu/services/training-infrastructure/