Fedcloud-tf:Testbed

From EGIWiki
Jump to: navigation, search
Main Roadmap and Innovation Technology For Users For Resource Providers Media

Contents



This page is for members of the Federated Cloud Task Force. If you wish to access resources from the EGI Federated Cloud, then please consult with the User Communities section


Technologies

The federation test bed does not mandate what VMM its resource providers should use. The federation adopts a set of well-defined functionalities and (standard) interfaces that every provider is free to implement independently.

[[Image:|pChart]]

Endpoints

The management interface endpoints made available by the TF Resource Providers

Provider Interface Type Endpoint
BIFI (UNIZAR)

Keystone (VOMS, user/pass)
Openstack EC2
Openstack Nova API
OCCI 1.1
LDAP Server

https://server4-epsh.unizar.es:5000/v2.0
http://server4-epsh.unizar.es:8773/services/Cloud
http://server4-epsh.unizar.es:8774/v2/
http://server4-epsh.unizar.es:8787
ldap://ce-iber.bifi.unizar.es:2170
BSC EMOTIVE OCCI+OVF
CDMI Proxy (user/pass)
CDMI Proxy (x509)
https://bscgrid20.bsc.es/DRP/compute/
http://bscgrid20.bsc.es:2365
https://bscgrid05.bsc.es:443
CC-IN2P3 Keystone
Openstack EC2
Openstack Nova API 1.1
Openstack S3
OCCI 1.1 (X509)
LDAP server
https://cckeystone.in2p3.fr:5000
http://ccec2.in2p3.fr:8773/services/Cloud
https://ccnovaapi.in2p3.fr:8774/v1.1/
http://ccs3.in2p3.fr:3333
https://ccocci.in2p3.fr:8787
ldap://cccldbdii01.in2p3.fr:2170.
CESGA OCCI 1.1 (VOMS/X.509 Auth)
LDAP server
https://cloud.cesga.es:3202
ldap://ui.egi.cesga.es:2170 -b o=glue
CESNET Sunstone
CDMI Proxy
LDAP server
OCCI 1.1 (X.509, VOMS)
https://carach5.ics.muni.cz/
https://carach3.ics.muni.cz:8080/
ldap://carach5.ics.muni.cz:2170
https://carach5.ics.muni.cz:11443/
Cyfronet OCCI 1.1 (X.509)
OCCI 1.1 (user/pass)
LDAP Server
OCCI 1.1 (rOCCI-0.5)
https://cloud-lab.grid.cyf-kr.edu.pl:3443/
http://cloud-lab.grid.cyf-kr.edu.pl:3200/
ldap://cloud-lab.grid.cyf-kr.edu.pl:2170
https://cloud-lab.grid.cyf-kr.edu.pl:11443/
FZ Jülich
  • OCCI 1.1 SSL
  • LDAP
  • Keystone (VOMS and User/Pass)
  • Swift
GRIF StratusLab https://onehost-2.lal.in2p3.fr
GRNET
  • occi (x509 voms)
  • Ldap Server
GWDG Sunstone
OCCI 0.8
OCCI 1.1 (user/pass)
OCCI 1.1 (X.509)
CDMI proxy (user/pass)
CDMI proxy (X.509)
LDAP Server
https://one.cloud.gwdg.de:8443
http://occi.cloud.gwdg.de:3400
http://occi.cloud.gwdg.de:3200
https://occi.cloud.gwdg.de:3100
http://cdmi.cloud.gwdg.de:4001
https://cdmi.cloud.gwdg.de:4000
ldap://one.cloud.gwdg.de:2170
CSIC Keystone (VOMS, user/pass)
Openstack EC2
Openstack Nova API 1.1
OCCI 1.1
LDAP Server
https://keystone.ifca.es:5000/v2.0
https://cloud.ifca.es/services/Cloud
http://cloud.ifca.es:8774/v1.1/
http://cloud.ifca.es:8787
ldap://cloud.ibergrid.eu:2170
IGI/INFN WNoDeS
OCCI 1.1(X509)
https://test-wnodes-web01.cnaf.infn.it:8443/
KTH OCCI 1.1 (x.509 auth)
LDAP
https://egi.cloud.pdc.kth.se:443/
ldap://egi.cloud.pdc.kth.se:2170/
SARA Sunstone
OCCI 0.8
OCCI 1.1 (X.509)
LDAP
http://ui.cloud.sara.nl/
https://occi.cloud.sara.nl/
https://occi11.cloud.sara.nl/
ldap://bdii.cloud.sara.nl:2170
TCD StratusLab OpenNebula proxy https://cagnode42.cs.tcd.ie:2634
ldap://cagnode42.cs.tcd.ie:2170
SZTAKI Sunstone
OCCI 1.1 (X.509)
OpenNebula EC2
OCCI 1.1 (X.509, rOCCI-server v0.5)
http://cfe2.lpds.sztaki.hu/
http://cfe2.lpds.sztaki.hu:4568/
http://cfe2.lpds.sztaki.hu:4567
https://cfe2.lpds.sztaki.hu:3333/
IISAS Openstack EC2
OCCI 1.1
Keystone (VOMS supported)
LDAP Server
http://nova.ui.savba.sk:8773
http://nova.ui.savba.sk:8787
https://keystone.ui.savba.sk:5000
ldap://nova.ui.savba.sk:2170
INFN-BARI OCCI 1.1
Keystone (VOMS supported)
LDAP Server
https://prisma-cloud.ba.infn.it:8787
https://prisma-cloud.ba.infn.it:5000
ldap://prisma-cloud.ba.infn.it:2170

Resource Providers inventory

The Resource Providers that have joined the Task Force make available a small portion of their cloud infrastructure in order to design and test the technologies described in the blueprint document for clouds federation. These resources are available for testing to every user community interested in testing/using them.

For the description of the Capabilities please refer to the Cloud Integration Profile document.

Please note: wherever suitable and possible, any standards implemented by the adopted cloud software should be noted.

Provider Status capacity Capabilities Management Interface Authentication
VM Management Data Information Monitoring Accounting Notification Supported Planned Service layer VMs

BIFI (UNIZAR) (Ruben Valles,

Jaime Ibar)

Integrating (Services ready to be tested) 360 cores( Xeon servers 2xhexacore 24GB RAM) OpenStack Grizzly

Local disks. Volume service CINDER.

SWIFT S3 compatible ( in future)

N/A Zabbix ( internal ) Apel SSM N/A EC2/Nova/OCCI



user/password, VOMS+Keystone

 Yet to be defined. Public images already available.

SSH keys


BSC
(Daniele Lezzi)
In production with VENUS-C middleware 96 cores (4 bi-processor Intel Xeon 6 cores, 24GB RAM; 3 bi-processor AMD Opteron 8 cores, 32GB RAM) Emotive Cloud (BSC), planned to move to OpenNebula and OpenStack Shared GlusterFS 3.6TB total; CDMI Proxy and FTP Emotive Cloud n/a VENUS-C Accounting system N/A OCCI+OVF OCCI provided by OpenNebula/OpenStack X.509 with VPN SSH keys
CESGA (IBERgrid)
(Ivan Diaz, Esteban Freire)
Production 33 octo-core servers (264 CPUs) OpenNebula 3.0 Shared NFS/SSH ~ 450GB per server OpenNebula/OCCI Ganglia In-house WIP Development N/A OCCI and partial EC2 provided by OpenNebula
Username and Password X.509 future As chosen by the users
CESNET-MetaCloud (CESNET)
(Miroslav Ruda)
Production 240 cores, 960GB RAM and 44TB shared storage OpenNebula 4.4.1 Shared NFS filesystem, GridFTP, S3 Cumulus, CDMI Proxy OpenNebula + OCCI v1.1 (rOCCI-server) Nagios infrastructure is ready, custom probes for OpenNebula's OCCI, ECONE, OCA. Ganglia / Munin can be added on request. OpenNebula accounting daemon + SSM LB notification + STOMP based EGI messaging infrastructure is available on the site OCCI v1.1 (rOCCI-server v1.0.0) Open for discussion Username and password, X.509 certificates for OCCI In general up to the user, currently registered SSH keys for root access to the VMs (cloud-init)
CETA-CIEMAT
(Abel Paz)
Testbed, under construction 14 servers (8 cores, 16 GB each one) OpenStack Essex Shared NFS filesystem N/A Nagios N/A Nagios notifications for admins (not users) EC2/Nova/OCCI Open for discussion Username and password, implemented by OpenStack SSH keys
CSIC
(Alvaro Lopez)
Production 32 x 8 core servers, 16GB RAM Openstack Grizzly Local disks. Volume service (cinder) available. n/a n/a n/a n/a EC2/OpenStack API/OCCI - user/password, VOMS SSH with user defined keys. Upload key through OCCI possible.
Cyfronet
(Tomasz Szepieniec, Marcin Radecki)

for initial setup 12 servers ready, extensions depending on usage Most likely OpenNebula 3.0 Possibility for mounting iSCSI devices in VMs, others to be defined Web interface integrated with PL-Grid User Portal Nagios integration, experimenting with zabbix Planned for early 2012 integration with PL-Grid Accounting using OpenNebula 3 accounting components N/A



FZ Jülich
(B. Hagemeier)
Testbed
4 Servers (76 Cores, 80GB RAM, 5TB Disk) OpenStack 'Folsom', Keystone 'Grizzly'

OpenStack Swift (CDMI i/f planned), Volume service (cinder)

n/a depends on solution above Nagios OpenStack + SSM n/a

Nova, OCCI


Username and password as implemented by OpenStack, VOMS proxy certificates for VOs 'fedcloud.egi.eu' and 'ops' User SSH keys for root access (configured via contextualization)
GRIF
(Michel Jouvin)
Production 10 servers (240 cores) StratusLab iSCSI-based permanent disks n/a n/a n/a n/a Private (StratusLab) OCCI X509 certificates preferred, username and password also possible User SSH keys for root access (configured when VM is launched)
GRNET
(Panos Louridas, Kostas Koumantaros)
Alpha 25 servers (200 cores, 48 GB RAM each server), 22 TB storage
Synefo (GRNET OpenStack implementation) Local disks

OpenStack compatible

voms enabled occi 1.1 interface 

Nagios, Munin, collectd, scripts In house development
OpenStack, also complete web based environment
Shibboleth, invitation tokens User SSH
GWDG
(Philipp Wieder)
Accessible October 23, 2011 As a start: 4 servers with Dual-Proc AMD Quad-Core "Barcelona", 2,4 GHz, 16 GB Ram, 250 GB HD. More beginning 2012 OpenNebula 3.2 with OCCI server Shared NFS OpenNebula Web interface (Sunstone) tbd (most likely Nagios) Currently n/a, usage of OpenNebula 3.2 accounting components planned for late 2011 n/a OCCI
Username and password, additionally X.509 in the future Up to the user, support for preregistered ssh keys in the future
IGI
(Giancinto Donvito, Paolo Veronesi)
A new dedicated testbed is under configuration. 24 cores, 48 GB RAM, 2TB Disk WNoDeS Shared NFS filesystem Usage of the Software­Run­Time­Environment attribute for publishing VM information by using BDII (work in progress) Nagios accounting at batch system level(pbs)and integrated with DGAS Accounting System used for the Grid infrastructure in Italy notification based on Nagios for system administrator (not for end users) OCCI CREAM Web Portal (authentication based on X509) expected in the next 2 months. Federated Single Sign-On Au­then­ti­ca­tion Service (based on Shibboleth) should be supported in the next 4/6 months. GSI (Grid Security Infrastructure based on X509 personal certificates and VO membership based on VOMS) SSH keys for root access
CC-IN2P3
(Helene Cordier, Gille Mathieu, Mattieu Puel)
Testbed 16 x (24 cores, 96GB RAM, 2TB local disk) = 384 cores Openstack Essex Local disks undef Nagios, Collectd/Smurf undef undef EC2/Nova/OCCI OCCI user/password, x509 when available OpenSSH
KTH
(Zeeshan Ali Shah)
Accessible since January, 2011 Initially 2 Servers with Total 4 cores, 16 GB RAM and 1TB storage OpenNebula Possibility to mount nfs storage OpenNebula Web interface with OCCI and OCA api Ganglia (need to experiment) N/A N/A OCCI Open for discussion Username and Password and X509  SSH Keys
OeRC (UK NGI)
(David Wallom, Matteo Turilli)

10 servers, between 8 and 2 VMs each Deploying OpenStack Data supplied through S3/EBS capable storage services N/A NAGIOS based Developed service utilising extended OGF UR schema N/A Partial EC2 as implemented by OpenStack OCCI when available Username and password as implemented by OpenStack As chosen by the users
SARA
(Jhon Masschelein, Maurice Bouwhuis, Machiel Jansen)
In production 1 January 2012 609 cores, 4,75 TB RAM OpenNebula 400 TB mountable storage, local disk 10 TB Web interface and Red Mine portal Nagios, Ganglia OpenNebula (adapted) Based on Nagios OCCI and partial EC2 provided by OpenNebula Open for discussion username password, X509 planned User defines
STFC
(Ian Collier)












TCD
(David O'Callaghan, Stuart Kenny)
Testing 5 x dual quad core with 16GB RAM StratusLab, OpenNebula Shared NFS filesystem, 1.5 TB StratusLab web-monitor, Sunstone Nagios n/a n/a StratusLab, OpenNebula
X509, Username and password User SSH key for root access
SZTAKI
(Sandor Acs, Peter Kotcauer, Mark Gergely)
Testing 128 cores, 308GB RAM OpenNebula 3.8.1 33TB (RAID5) iSCSI/AoE storage + local storages (~10TB) OpenNebula OCCI/ECONE Nagios, Munin, Zabbix OpenNebula (adapted) N/A OCCI (both of standard OpenNebula and rOCCI) and partial EC2 provided by OpenNebula Open for discussion Username and password, X.509 certificates for OCCI SSH with password or with user key are preferred
IISAS
(Viet Tran, Binh Minh Nguyen)
Testing Initially 2 servers with 16 cores, 48GB RAM, extension after testing OpenStack Folsom Shared NFS N/A N/A (Nagios planned) N/A N/A EC2/Nova/OCCI Open for discussion user name/password, x509 being tested SSH keys
100 Percent IT Ltd
(David Blundell)
Testing 24 cores, 64GB RAM, 4TB disk OpenStack Grizzly S3/EBS capable storage service Web interface undef undef undef EC2/Nova/OCCI Open for discussion Username and password as implemented by OpenStack User defined

Technology Provider inventory

The Technology Providers of the Task Force offer support for the technologies that they develop and evaluate further development in accordance with the federation roadmap.

For the description of the Capabilities please refer to the Cloud Integration Profile document.

Please note: wherever suitable and possible, any standards implemented by the adopted cloud software should be noted.

Provider Capabilities
VM Management Data Information Monitoring Accounting Notification
StratusLab (Cal Loomis) OpenNebula using XML-RPC interface (eventually OCCI); standard OpenNebula VM description for files (eventually OVF); authentication options are username and password, grid certificates and VOMS proxies, others methods should be easy to add Proprietary Persistent Disk Store with RESTful interface (eventually also CDMI) Planned in architecture, not implemented Planned in architecture, not implemented Some functionality in OpenNebula, implementation for all StratusLab services planned but not yet implemented Prototype implementation in place, allows notification through AMQP if users provide messaging coordinates when starting a virtual machine
EGI-InSPIRE JRA1 (Daniele Cesini) None None None

EGI-JRA1 offers NAGIOS probes integration Capability - No NAGIOS probes development is foreseen within JRA1 (with the exception of very few cases) - Technology Providers are expected to produce NAGIOS probes for their own systems.

Availability and Reliability calculation and reporting for sites and services are currently produced outside EGI-JRA1.

No information discovery systems are developed within JRA1.

EGI-JRA1 contains a task (TJRA1.4) responsible for the development of an accounting system capable of encompassing the new resource types that will appear in the EGI infrastructure including virtualised resources. See the EGI DoW for TJRA1.4 details. None
WNoDeS (Davide Salomoni, Elisabetta Ronchieri) WNoDeS, with OCCI interface Posix I/O planned on Lustre, NFS and GPFS as persistent storage Usage of the Software­Run­Time­Environment attribute for publishing VM information by using BDII Internal monitoring system for hypervisors. Not yet integrated with NAGIOS probes Accounting at batch system level (like lsf and pbs) and integration with the DGAS Accounting System used by the Italian Grid infrastructure


Cloud Resources Status

The Task Force is developing a resource monitor solution for the clouds federation based on Nagios. Meanwhile, here a table showing the current status of the cloud resources made available by the resource providers that have joined the Task Force. This table is updated weekly by the resource providers.

Providers

    = Available
    = Not available

User registration User access VM availability Elastic IPs Object Storage Persistent Storage
CESGA (IBERgrid)





CESNET (NGI CZ)
[1]

Sunstone OCCI v0.8 OCCI v1.1



debian6

Yes

Cumulus at carach3.ics.muni.cz:8888

GridFTP at carach4.ics.muni.cz:50000
CYFRONET (NGI PL)





GWDG [2]                         
CSIC (Ibergrid)
                        
FZ Jülich Members of VO fedcloud.egi.eu and ops are allowed     Yes   Swift
    
IGI





CC-IN2P3 (NGI FR) CC-IN2P3 account request                         
KTH [3]          
         
OerC (UK NGI)





SARA (NGI NL) [4] ui.cloud.sara.n                    
TCD (NGI IE)





SZTAKI (NGI HU) [5]                         
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Print/export