Difference between revisions of "ELIXIR Virtual Organisation"
Latest revision as of 12:48, 3 July 2017
|EGI-Engage Competence centres:||Main page||ELIXIR||BBMRI||MoBrain||DARIAH||LifeWatch||EISCAT_3D||EPOS||Disaster Mitigation |||EGI-Engage Knowledge Commons|
This Virtual Organisation (VO) contributes to the ELIXIR effort of building a sustainable European infrastructure for biological information, supporting life science research and its translation to medicine, agriculture, bioindustries and society. The Virtual Organisation constitutes the backbone of the ELIXIR Compute Platform and federates cloud compute and storage resources from ELIXIR and EGI providers.
The VO is currently open for the application piloting activities of the ELIXIR Competence Centre of the EGI-Engage H2020 project. Further information about the Competence Centre is available at https://wiki.egi.eu/wiki/CC-ELIXIR. Access to the VO resources is restricted to those working on the Competence Centre activities.
Cloud resources in the VO
- CESNET-MetaCloud: OpenNebula cloud; Local site admin contact: firstname.lastname@example.org
- IN2P3-IRES: OpenStack cloud; Local site admin contact: email@example.com
- EMBL-EBI: OpenStack cloud: Local site admin contact: Charles Short (<firstname.lastname@example.org>)
Under finalisation: GRNET Under discussion: CSC, SURFsara
- Steven Newhouse (EMBL-EBI)
- Miroslav Ruda (CESNET)
VO ID card in the EGI Operations Portal
Acceptable use policy
This Acceptable Use Policy applies to all members of the "vo.elixir-europe.org" Virtual Organisation, hereafter referred to as the VO, and the resources that members are able to access through the VO mechanism. Members of the EGI-Engage Competence Centre (https://wiki.egi.eu/wiki/CC-ELIXIR) owns and gives authority to this policy. This VO contributes to the ELIXIR effort of building a sustainable European infrastructure for biological information, supporting life science research and its translation to medicine, agriculture, bioindustries and society.
All VO members (users, managers, infrastructure providers) agree to be bound by this Acceptable Use Policy and to use the resources within the VO only in the furtherance of the stated goal of the VO. By registering in the VO as a user you shall be deemed to accept these conditions of use:
- You shall only use the VO services to perform work, or transmit or store data consistent with the stated goals, policies and conditions of use as defined by the body or bodies granting you access.
- You shall not use the VO for any unlawful purpose and not (attempt to) breach or circumvent any administrative or security controls.
- You shall respect intellectual property and confidentiality agreements.
- You shall protect your access credentials (e.g. passwords or private keys).
- You shall immediately report any known or suspected security breach or misuse of the VO or access credentials to email@example.com and to the relevant credential issuing authorities. (firstname.lastname@example.org for ELIXIR accounts)
- You must notify the Registrar of any changes to your Registration Information.
- Use of the VO is at your own risk. There is no guarantee that the VO will be available at any time or that it will suit any purpose.
- Logged information, including information provided by you for registration purposes, is used for administrative, operational, accounting, monitoring and security purposes only. This information may be disclosed, via secured mechanisms, only for the same purposes and only as far as necessary to other organisations cooperating with the VO. Although efforts are made to maintain confidentiality, no guarantees are given.
- The access-granting bodies and Resource Providers are entitled to regulate, suspend or terminate your access, within their domain of authority, and you shall immediately comply with their instructions.
- You are liable for the consequences of you violating any of these conditions of use.
- The VO includes core services from the EGI e-infrastructure, thus other relevant EGI Policies and Procedures also apply to certain VO member groups. See these policies at http://www.egi.eu/about/policy/policies_procedures.html. (Policies tagged 'Users' apply to VO members, Policies tagged 'Infrastructure' apply to cloud providers)
Instruction for users
How to register
- Apply for VO membership at https://perun.elixir-czech.cz/registrar/?vo=elixir&group=EGI:vo.elixir-europe.org. You will be asked to create an account in ELIXIR, this is part of the registration process to the VO vo.elixir-europe.org.
- VO membership is received and evaluated by the VO Managers. Membership is currently restricted to those working in the ELIXIR Competence Centre application/service porting activities. Membership is expected to be broadened for other ELIXIR partners during 2017.
- You receive a notification email about the approval/rejection of your VO membership request.
How to use IaaS clouds
GUI: AppDB VMOps Dashboard
The EGI Application Database (AppDB) has recently evolved its functionalities from its catalogue of applications and virtual machines (VMs) to include a Graphical User Interface (GUI) to perform VM management operations on the distributed infrastructure.
Follow the VMOps Dashboard guide to get more information about its usage. You should be able to access it with login in via EGI CheckIn and selecting ELIXIR as identity provider.
API and CLI
Resources available on the ELIXIR VO are accessed using X.509 proxy certificates with VOMS extensions (claims stating the membership of the user to the VO). You can use your ELIXIR identity to get one valid proxy by interacting with the CILogin service, currently there are two methods:
- Creating a client for the CILogon service, check the example at https://github.com/elixirhub/elixir-aai/tree/master/cilogon
- Using CILogon/RCAuth.eu certs on demo VO portal
Once you have a proxy, you need to add the VOMS extensions. This can be done with voms-proxy-init with the
--noregen option, for example (this copies first the proxy to the default location so you don't overwrite you original one):
cp your_proxy /tmp/x509up_u$(id -u) chmod 600 /tmp/x509up_u$(id -u) voms-proxy-init --noregen --rfc --voms vo.elixir-europe.org
OCCI and OpenStack access
IaaS cloud resources can expose two types of interfaces towards users (one or the other or both - depending on the cloud provider):
- Open Standard interfaces: OCCI (Open Cloud Computing Interface) to manage compute, blocks storage and network resources. This interface set are currently exposed by all of the OpenNebula and Synnefo cloud providers, and some of the OpenStack providers.
- OpenStack interfaces: The native OpenStack interfaces (with X.509 authentication). These interfaces are currently exposed by the OpenStack-based EGI cloud providers.
The user can interact with IaaS cloud resources via programming APIs and command line interfaces. Web dashboard access and Ansible orchestrator access are currently under development. The different access modes are summarized in the following table:
|Open Standards interface||OpenStack interface|
|API level access||OCCI||OpenStack Compute & Openstack Object Storage|
|Command Line access||rOCCI-cli||OpenStack CLI with VOMS authentication plugin|
|Web dashboard access||AppDB VMOps Dashboard||OpenStack Horizon (in tests)|
Check out these tutorial slides for a practical overview on how to use the IaaS resources using the rOCCI-cli. The slides cover the following topics:
- Using pre-defined Virtual Machine images
- Customised VM deployments (contextualisation)
- Docker containers in the EGI cloud
- Preparing your own VM image (with Packer)
Native OpenStack with EGI-OpenStack-Terraform
EGI provides a Terraform provider plugin that extends the builtin OpenStack provider of OpenStack with support for EGI AAI. Documentation on how to install and use is avaiable at Federated Cloud IaaS Orchestration page
Native OpenStack with tokens
Terraform has OpenStack support out of the box, however it does not support the X.509 based authentication of EGI. Instead you can use token based authentication. Tokens normally have a lifetime of 1 hour, if your deployment last longer you should check the EGI OpenStack Terraform plugin . For obtaining such token, you can follow these steps:
- Install the OpenStack CLI, there are several ways of getting this, but an easy way is just using pip, you can use a virtualenv to ensure your regular python environment is intact:
$ pip install python-openstackclient
- Install the VOMS authorisation plugin
$ pip install openstack-voms-auth-type
- Get a list of projects where you are allowed (in this example, using the EBI access point):
$ openstack --os-auth-url https://extcloud04.ebi.ac.uk:5000/v2.0 --os-auth-type v2voms \ --os-x509-user-proxy /tmp/x509up_u1000 project list +----------------------------------+--------+ | ID | Name | +----------------------------------+--------+ | e99a879a2d9e4b01b9152637c7bde4cb | elixir | +----------------------------------+--------+
In case of getting SSL errors, check the CA Certificates information for OpenStack CLI
- With the project you can get a token:
$ export OS_AUTH_TOKEN=$(openstack --os-auth-url https://extcloud04.ebi.ac.uk:5000/v2.0 \ --os-auth-type v2voms \ --os-x509-user-proxy /tmp/x509up_u1000 \ --os-project-id e99a879a2d9e4b01b9152637c7bde4cb \ token issue -c id -f value)
OS_AUTH_TOKEN variable will be used by Terraform if available in the environment so you don't have to include in your
Terraform with OCCI
There is a OCCI plugin for Terraform developed by CESNET and available at GitHub: https://github.com/cduongt/terraform/tree/occi.
Installation of this plugin requires compilation, check the README file for specific information. The plugin allows to manage VMs at OCCI endpoints
Joining the VO as IaaS cloud provider
The VO welcomes further IaaS cloud providers. D6.10 deliverable of the ELIXIR Competence Centre provides guidance for cloud providers on how can an IaaS cloud federate into the VO: https://documents.egi.eu/document/2841. Technology currently exist to federate OpenStack, OpenNebula and Synnefo cloud management framework based cloud sites.
Please write to <email@example.com> to express your interest in joining the VO as a cloud provider.