Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Fedcloud-tf:CloudscapeVDemo"

From EGIWiki
Jump to navigation Jump to search
 
(13 intermediate revisions by the same user not shown)
Line 9: Line 9:
# Check Nagios/SAM for status
# Check Nagios/SAM for status
## --> demonstrates monitoring
## --> demonstrates monitoring
## https://fedcloud-mon.egi.eu
## http://cloudmon.egi.eu/myegi/
# Look up the image on the Marketplace
# Look up the image on the Marketplace
## --> demonstrate VM distribution and endorsement
## --> demonstrate VM distribution and endorsement
## --> demonstrate vmcatcher based image distribution
## http://marketplace.egi.eu/metadata
## http://marketplace.egi.eu/metadata
# using rOCCI client, deploy a number of WeNMR instance on selected RPs
# using rOCCI client, deploy a number of WeNMR instance on selected RPs
Line 22: Line 23:
## http://goc-accounting.grid-support.ac.uk/cloudtest/vmshour.html
## http://goc-accounting.grid-support.ac.uk/cloudtest/vmshour.html
## http://goc-accounting.grid-support.ac.uk/cloudtest/vms.html
## http://goc-accounting.grid-support.ac.uk/cloudtest/vms.html


= TODOs =
= TODOs =


== rOCCI client ==
== <span style="color:green">rOCCI client</span> ==


====<span style="color:green">DONE</span> Get rOCCI client====
====<span style="color:green">Get rOCCI client</span> ====


This was quite a hassle for MAC OS X 10.8 (Mountain Lion). Here is what I did - including a WARNING that I might have to re-compile Ruby 1.9.3.
This was quite a hassle for MAC OS X 10.8 (Mountain Lion). I finally managed getting rOCCI client running natively on Mac OS X - [[rOCCI client on Mac OS X Mountain Lion (10.8)|here is how I did it]].
* Install Xcode 4.6 (from developer.appe.com
* Inside Xcode, download and install Xcode command line tools (Menu Xcode - Preferences - Downloads)
* Install RVM<br>Jewlery crashed on my system, so that was a dead end. The following command tried to compile Ruby 1.9.3 but failed with a warning on an unsupported CLANG feature. Looks like the gcc I used is the gcc-llvm from Apple... But tht at least installed RVM on my system...
$ curl -L https://get.rvm.io | bash -s stable --ruby
* Read up on some discussions on this here: http://stackoverflow.com/questions/14592945/cannot-compile-ruby-1-9-3
* Try again compiling Ruby with a different approach (mentioned in that thread. That compiled Ruby 1.9.3 fine, though with the warning on CLANG. So I am still using gcc-llvm
$ rvm install 1.9.3 --with-gcc=gcc
* Installation went fine... So now on to installing OCCI...
gem install occi
[...]
Successfully installed antlr3-1.8.12
Successfully installed hashie-1.2.0
Successfully installed uuidtools-2.1.3
Successfully installed nokogiri-1.5.6
Successfully installed i18n-0.6.1
Successfully installed multi_json-1.5.0
Successfully installed activesupport-3.2.11
Successfully installed multi_xml-0.5.2
Successfully installed httparty-0.10.2
Successfully installed highline-1.6.15
Successfully installed eventmachine-1.0.0
Successfully installed amq-protocol-1.1.0
Successfully installed amq-client-0.9.11
Successfully installed amqp-0.9.8
Successfully installed occi-3.0.0
15 gems installed
[...]
$
Done! Does it blend? ;-)
$ occi --auth x509 --action list --resource storage --user-cred /Users/michel/.globus/usercred-des.pem --ca-path /Users/michel/FCTF/certificates --endpoint https://occi.cloud.gwdg.de:3100/
Enter a password or an auth. token:
line 1:145 mismatched character "u"; expecting "k"
line 1:157 mismatched character "u"; expecting "k"
Storage locations:
https://occi.cloud.gwdg.de:3100/storage/72666675-9e33-55c7-8205-b157e3c8e580
https://occi.cloud.gwdg.de:3100/storage/ea4f06a8-3509-5cbf-9f3e-f76917d60e7d
https://occi.cloud.gwdg.de:3100/storage/ed731887-2cd1-56ba-8f7c-1fa9ded7280e
https://occi.cloud.gwdg.de:3100/storage/9167833e-f720-5e94-a884-3911a58fa127
https://occi.cloud.gwdg.de:3100/storage/5778be67-1da0-55e0-99c9-469060e65d5c
https://occi.cloud.gwdg.de:3100/storage/28a83f58-77a0-5424-b0bf-47bb54f39120
https://occi.cloud.gwdg.de:3100/storage/2ebee24b-b268-5389-a22a-ea15a4cc8003
https://occi.cloud.gwdg.de:3100/storage/62e31e68-fd09-5669-ae0c-e7eb1f783bb4
https://occi.cloud.gwdg.de:3100/storage/a6b0bd70-8f42-5d9b-94ec-a6e0429f24e1
https://occi.cloud.gwdg.de:3100/storage/fe61d5dc-174e-509b-aa5d-3a775e46945a


It blends! And it even works out of the box with encrypted private keys in PEM format (see commandline options above)
<span style="color:red">Action rOCCI team</span> - document usage on Mac OS X (issue filed on Github) (see https://github.com/gwdg/rOCCI/issues/59)


<span style="color:red">Action rOCCI team</span> - document usage on Mac OS X (issue filed on Github)
====<span style="color:green">rOCCI client and my Grid certificate </span>====
 
====<span style="color:green">DONE</span> rOCCI client and my Grid certificate ====


After sorting out the issues with Ruby and rOCCI client (see above), the certificate handling wasn't a problem anymore. However, I stand with my request to support PKCS#12 format for key and cert storage as this is *very* widely supported and does not need people to mess with OpenSSL cmd line hacking to get the authN sorted.  
After sorting out the issues with Ruby and rOCCI client (see above), the certificate handling wasn't a problem anymore. However, I stand with my request to support PKCS#12 format for key and cert storage as this is *very* widely supported and does not need people to mess with OpenSSL cmd line hacking to get the authN sorted.  
Line 90: Line 42:
<span style="color:red">Action</span> - Please document '''in easy steps''' the whole process of acquiring a Grid certificate and how to configure command line systems (for PEM support)
<span style="color:red">Action</span> - Please document '''in easy steps''' the whole process of acquiring a Grid certificate and how to configure command line systems (for PEM support)


<span style="color:red">Action rOCCI team</span> - Support PKCS#12 (https://github.com/gwdg/rOCCI/issues/48)
==== <span style="color:Green;">Getting started with client</span> ====


==== <span style="color:Orange;">In progress</span> Getting started with client ====
Set up a couple of config scripts that make the command line very easy. See raw demo takes available at https://documents.egi.eu/document/1593)


Getting familiar with it. For demo purposes, long command line options are a killer, so some sort of configuration file with endpoint profiling would be good.
For longer term, a rOCCI client config file will be helpful.


<span style="color:red">Action rOCCI team</span> - Support config files with endpoint profiling (https://github.com/gwdg/rOCCI/issues/46)
<span style="color:red">Action rOCCI team</span> - Support config files with endpoint profiling (https://github.com/gwdg/rOCCI/issues/46)


== RP status ==
== <span style="color:Orange;">RP status</span> ==
 
==== <span style="color:Green;">Getting started with client</span> ====
 
See  rOCCI client configuration. Endpoint queries are stable and promising. Focusing on OCCI endpont queries.


==== <span style="color:Orange;">In progress</span> Getting started with client ====
==== <span style="color:Orange;">Compute queries</span> ====


Checking RP status for the demo. Checks include:
The OCCI command is (omitting authentication parameters):
* storage resource query checks against RPs '''without''' VO support
  occi --endpoint $ENDPOINT --resource compute --action list
The OCCI command used to query the resource providers is this:
 
  occi --auth x509 --user-cred /Users/michel/.globus/usercred-des.pem --ca-path /Users/michel/FCTF/certificates --password $PASSWD --action list --resource storage  --endpoint $ENDPOINT
RP status<br>
Status:<br>
<span style="color:Green;">CESGA, CESNET, GWDG, GRIF</span><br>
<span style="color:Orange;">JUELICH, GRNET, IN2P3, IFCA</span> all authentication errors<br>
<span style="color:Red;">CYFRONET, INFN</span> both no endpoint in GOCDB) <br>
 
==== <span style="color:Orange;">Compute instantiations</span> ====
The OCCI command is (omitting authentication parameters):
occi --endpoint $ENDPOINT --resource compute --action create --mixin os_tpl#$TPL -t "title"
 
RP status<br>
<span style="color:Green;">CESGA, CESNET, GWDG</span><br>
<span style="color:Green;">CESGA, CESNET, GWDG</span><br>
<span style="color:Orange;">Cyfronet (no endpoint in GOCDB), INFN (no endpoint in GOCDB)</span><br>
<span style="color:Orange;">GRIF</span> Fails to return resource identifier, no resource creatred<br>
<span style="color:Red;">JUELICH (credentials rejected?), IN2P3 (service timeout)</span><br>
<span style="color:Orange;">JUELICH, GRNET, IN2P3, IFCA</span> all authentication errors<br>
* storage resource queries against RPs '''with''' VO support
<span style="color:Red;">CYFRONET, INFN</span> both no endpoint in GOCDB) <br>
* instantiate WeNMR image '''without VO support'''
The OCCI command used to query the resource providers is this:
occi --auth x509 --user-cred /Users/michel/.globus/usercred-des.pem --ca-path /Users/michel/FCTF/certificates --password $PASSWD --endpoint $ENDPOINT --resource compute --action create --mixin $MIXIN --resource-title "HelloWeNMR"
Status:<br>
<span style="color:Green;">CESGA, CESNET</span><br>
<span style="color:Orange;">Cyfronet (no endpoint in GOCDB), INFN (no endpoint in GOCDB), JUELICH (credentials rejected?), IN2P3 (service timeout)</span><br>
<span style="color:Red;">GWDG ("Message: wrong constant name" for --mixin os_tpl#vcing)</span><br>


== VO support ==


==== <span style="color:Green;">Done</span> Get my Grid Certificate registered in Fedcloud VO ====


Server Perun doesn't like me. CESNET is investigating.
==  <span style="color:Green;">GOC DB </span>==


Update - tried this morning again, and I could apply for fedcloud membership. Waiting for confirmation Email...
=====GocDB CA certificate =====


==== <span style="color:Orange;">InProgress</span> Get Proxy certificate tools ====
Add UK E Science certificate from the trust bundle into my browser config. For Mac OS X that translates to key chain access fun (for Google Chrome)


No idea yet...
==  <span style="color:Green;"> SAM / Monitoring </span> ==


==== Test-run OCCI commands against selected RPs ====
The SAM instance for FCTF is used: https://cloudmon.egi.eu/nagios


== SAM / Monitoring ==
My certificate is recognised, and the demo part for SAM is taken.


Make sure the production SAM is used: https://fedcloud-mon.egi.eu
== <span style="color:Green;">Accounting</span> ==


== Accointing ==
Done. Will take it, and cute the video to time shift.


== GOC DB ==
== VO support ==


===== <span style="color:Green;">DONE</span> GocDB CA certificate =====
No VO support for Cloudscape.


Add UK E Science certificate from the trust bundle into my browser config. For Mac OS X that translates to key chain access fun (for Google Chrome)


[[Category:Technology]] [[Category:Fedcloud-tf]]
[[Category:Technology]] [[Category:Fedcloud-tf]]

Latest revision as of 13:36, 20 February 2013


This Wiki entry describes the planned demonstration at Cloudscape V in Brussels (see FCTF Outreach section).

Demonstration script

  1. Check GOGDB for available Cloud endpoints
    1. --> demonstrates information system
    2. https://goc.egi.eu/portal/
  2. Check Nagios/SAM for status
    1. --> demonstrates monitoring
    2. http://cloudmon.egi.eu/myegi/
  3. Look up the image on the Marketplace
    1. --> demonstrate VM distribution and endorsement
    2. --> demonstrate vmcatcher based image distribution
    3. http://marketplace.egi.eu/metadata
  4. using rOCCI client, deploy a number of WeNMR instance on selected RPs
    1. --> federated consistent access using OCCI
    2. tentative RPs: INFN, JUELICH, GWDG, CESNET, CESGA, Cyfronet, In2P3
    3. refer to WeNMR page for info on this demo phase
  5. Go to the accounting page
    1. --> demonstrate that we account for Cloud compute consumption
    2. http://goc-accounting.grid-support.ac.uk/cloudtest/cloudsites.html
    3. http://goc-accounting.grid-support.ac.uk/cloudtest/vmshour.html
    4. http://goc-accounting.grid-support.ac.uk/cloudtest/vms.html

TODOs

rOCCI client

Get rOCCI client

This was quite a hassle for MAC OS X 10.8 (Mountain Lion). I finally managed getting rOCCI client running natively on Mac OS X - here is how I did it.

Action rOCCI team - document usage on Mac OS X (issue filed on Github) (see https://github.com/gwdg/rOCCI/issues/59)

rOCCI client and my Grid certificate

After sorting out the issues with Ruby and rOCCI client (see above), the certificate handling wasn't a problem anymore. However, I stand with my request to support PKCS#12 format for key and cert storage as this is *very* widely supported and does not need people to mess with OpenSSL cmd line hacking to get the authN sorted.

Any browser and OS key management supports PKCS#12 key management - no command line private key messing involved!

Action - Please document in easy steps the whole process of acquiring a Grid certificate and how to configure command line systems (for PEM support)

Getting started with client

Set up a couple of config scripts that make the command line very easy. See raw demo takes available at https://documents.egi.eu/document/1593)

For longer term, a rOCCI client config file will be helpful.

Action rOCCI team - Support config files with endpoint profiling (https://github.com/gwdg/rOCCI/issues/46)

RP status

Getting started with client

See rOCCI client configuration. Endpoint queries are stable and promising. Focusing on OCCI endpont queries.

Compute queries

The OCCI command is (omitting authentication parameters):

occi --endpoint $ENDPOINT --resource compute --action list

RP status
CESGA, CESNET, GWDG, GRIF
JUELICH, GRNET, IN2P3, IFCA all authentication errors
CYFRONET, INFN both no endpoint in GOCDB)

Compute instantiations

The OCCI command is (omitting authentication parameters):

occi --endpoint $ENDPOINT --resource compute --action create --mixin os_tpl#$TPL -t "title"

RP status
CESGA, CESNET, GWDG
GRIF Fails to return resource identifier, no resource creatred
JUELICH, GRNET, IN2P3, IFCA all authentication errors
CYFRONET, INFN both no endpoint in GOCDB)


GOC DB

GocDB CA certificate

Add UK E Science certificate from the trust bundle into my browser config. For Mac OS X that translates to key chain access fun (for Google Chrome)

SAM / Monitoring

The SAM instance for FCTF is used: https://cloudmon.egi.eu/nagios

My certificate is recognised, and the demo part for SAM is taken.

Accounting

Done. Will take it, and cute the video to time shift.

VO support

No VO support for Cloudscape.