Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

ELIXIR Virtual Organisation

From EGIWiki
Jump to navigation Jump to search
EGI-Engage Competence centres: Main page ELIXIR BBMRI MoBrain DARIAH LifeWatch EISCAT_3D EPOS Disaster Mitigation | EGI-Engage Knowledge Commons


Introduction

This Virtual Organisation (VO) contributes to the ELIXIR effort of building a sustainable European infrastructure for biological information, supporting life science research and its translation to medicine, agriculture, bioindustries and society. The Virtual Organisation constitutes the backbone of the ELIXIR Compute Platform and federates cloud compute and storage resources from ELIXIR and EGI providers.

The VO is currently open for the application piloting activities of the ELIXIR Competence Centre of the EGI-Engage H2020 project. Further information about the Competence Centre is available at https://wiki.egi.eu/wiki/CC-ELIXIR. Access to the VO resources is restricted to those working on the Competence Centre activities.

Cloud resources in the VO

  • CESNET-MetaCloud: OpenNebula cloud; Local site admin contact: cloud@metacentrum.cz
  • IN2P3-IRES: OpenStack cloud; Local site admin contact: grid.admin@iphc.cnrs.fr
  • EMBL-EBI: OpenStack cloud: Local site admin contact: Charles Short (<cems@ebi.ac.uk>)

Under finalisation: GRNET Under discussion: CSC, SURFsara

VO Managers

  • Steven Newhouse (EMBL-EBI)
  • Miroslav Ruda (CESNET)

VO ID card in the EGI Operations Portal

Acceptable use policy

This Acceptable Use Policy applies to all members of the "vo.elixir-europe.org" Virtual Organisation, hereafter referred to as the VO, and the resources that members are able to access through the VO mechanism. Members of the EGI-Engage Competence Centre (https://wiki.egi.eu/wiki/CC-ELIXIR) owns and gives authority to this policy. This VO contributes to the ELIXIR effort of building a sustainable European infrastructure for biological information, supporting life science research and its translation to medicine, agriculture, bioindustries and society.

All VO members (users, managers, infrastructure providers) agree to be bound by this Acceptable Use Policy and to use the resources within the VO only in the furtherance of the stated goal of the VO. By registering in the VO as a user you shall be deemed to accept these conditions of use:

  1. You shall only use the VO services to perform work, or transmit or store data consistent with the stated goals, policies and conditions of use as defined by the body or bodies granting you access.
  2. You shall not use the VO for any unlawful purpose and not (attempt to) breach or circumvent any administrative or security controls.
  3. You shall respect intellectual property and confidentiality agreements.
  4. You shall protect your access credentials (e.g. passwords or private keys).
  5. You shall immediately report any known or suspected security breach or misuse of the VO or access credentials to abuse@egi.eu and to the relevant credential issuing authorities. (aai-contact@elixir-europe.org for ELIXIR accounts)
  6. You must notify the Registrar of any changes to your Registration Information.
  7. Use of the VO is at your own risk. There is no guarantee that the VO will be available at any time or that it will suit any purpose.
  8. Logged information, including information provided by you for registration purposes, is used for administrative, operational, accounting, monitoring and security purposes only. This information may be disclosed, via secured mechanisms, only for the same purposes and only as far as necessary to other organisations cooperating with the VO. Although efforts are made to maintain confidentiality, no guarantees are given.
  9. The access-granting bodies and Resource Providers are entitled to regulate, suspend or terminate your access, within their domain of authority, and you shall immediately comply with their instructions.
  10. You are liable for the consequences of you violating any of these conditions of use.
  11. The VO includes core services from the EGI e-infrastructure, thus other relevant EGI Policies and Procedures also apply to certain VO member groups. See these policies at http://www.egi.eu/about/policy/policies_procedures.html. (Policies tagged 'Users' apply to VO members, Policies tagged 'Infrastructure' apply to cloud providers)

Instruction for users

How to register

  1. Apply for VO membership at https://perun.elixir-czech.cz/registrar/?vo=elixir&group=EGI:vo.elixir-europe.org. You will be asked to create an account in ELIXIR, this is part of the registration process to the VO vo.elixir-europe.org.
  2. VO membership is received and evaluated by the VO Managers. Membership is currently restricted to those working in the ELIXIR Competence Centre application/service porting activities. Membership is expected to be broadened for other ELIXIR partners during 2017.
  3. You receive a notification email about the approval/rejection of your VO membership request.

How to use IaaS clouds

Authentication

Resources available on the ELIXIR VO are accessed using X.509 proxy certificates with VOMS extensions (claims stating the membership of the user to the VO). You can use your ELIXIR identity to get one valid proxy by interacting with the CILogin service, currently there are two methods:

Once you have a proxy, you need to add the VOMS extensions. This can be done with voms-proxy-init with the --noregen option, for example (this copies first the proxy to the default location so you don't overwrite you original one):

cp your_proxy /tmp/x509up_u$(id -u)
chmod 600 /tmp/x509up_u$(id -u)
voms-proxy-init --noregen --rfc --voms vo.elixir-europe.org

OCCI and OpenStack access

IaaS cloud resources can expose two types of interfaces towards users (one or the other or both - depending on the cloud provider):

  • Open Standard interfaces: OCCI (Open Cloud Computing Interface) to manage compute, blocks storage and network resources. This interface set are currently exposed by all of the OpenNebula and Synnefo cloud providers, and some of the OpenStack providers.
  • OpenStack interfaces: The native OpenStack interfaces (with X.509 authentication). These interfaces are currently exposed by the OpenStack-based EGI cloud providers.

The user can interact with IaaS cloud resources via programming APIs and command line interfaces. Web dashboard access and Ansible orchestrator access are currently under development. The different access modes are summarized in the following table:

Open Standards interface OpenStack interface
API level access OCCI OpenStack Compute && Openstack Object Storage
Command Line access rOCCI-cli OpenStack CLI with VOMS authentication plugin
Web dashboard access AppDB VMOps Dashboard (in final test) OpenStack Horizon (in final tests)
Orchestrator access

Known options:

Known options:

  • Terraform (See below)

Check out these tutorial slides for a practical overview on how to use the IaaS resources using the rOCCI-cli. The slides cover the following topics:

  • Using pre-defined Virtual Machine images
  • Customised VM deployments (contextualisation)
  • Docker containers in the EGI cloud
  • Preparing your own VM image (with Packer)

Terraform orchestrator

Native OpenStack

Terraform has OpenStack support out of the box, however it does not support the X.509 based authentication of EGI. Instead you can use token based authentication. Tokens normally have a lifetime of 1 hour. For obtaining such token, you can follow these steps:

  • Install the OpenStack CLI, there are several ways of getting this, but an easy way is just using pip, you can use a virtualenv to ensure your regular python environment is intact:
$ pip install python-openstackclient
  • Install the VOMS authorisation plugin
$ pip install openstack-voms-auth-type
  • Get a list of projects where you are allowed (in this example, using the EBI access point):
$ openstack --os-auth-url https://extcloud04.ebi.ac.uk:5000/v2.0 --os-auth-type v2voms \
            --os-x509-user-proxy /tmp/x509up_u1000 project list
+----------------------------------+--------+
| ID                               | Name   |
+----------------------------------+--------+
| e99a879a2d9e4b01b9152637c7bde4cb | elixir |
+----------------------------------+--------+ 

In case of getting SSL errors, check the CA Certificates information for OpenStack CLI

  • With the project you can get a token:
$ export OS_AUTH_TOKEN=$(openstack --os-auth-url https://extcloud04.ebi.ac.uk:5000/v2.0 --os-auth-type v2voms \
                                   --os-x509-user-proxy /tmp/x509up_u1000 --os-project-id e99a879a2d9e4b01b9152637c7bde4cb \
                                   token issue -c id -f value)

The OS_AUTH_TOKEN variable will be used by Terraform if available in the environment so you don't have to include in your .tf file.

Terraform with OCCI

There is a OCCI plugin for Terraform developed by CESNET and available at GitHub: https://github.com/cduongt/terraform/tree/occi. Currently it supports management of VMs, however is lacking a way of specifying network information so it may not work for all available sites. There is ongoing discussion on how to overcome this issue.


Joining the VO as IaaS cloud provider

The VO welcomes further IaaS cloud providers. D6.10 deliverable of the ELIXIR Competence Centre provides guidance for cloud providers on how can an IaaS cloud federate into the VO: https://documents.egi.eu/document/2841. Technology currently exist to federate OpenStack, OpenNebula and Synnefo cloud management framework based cloud sites.

Please write to <cc-elixir@mailman.egi.eu> to express your interest in joining the VO as a cloud provider.