Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI Federated Cloud jOCCI APIs

From EGIWiki
Jump to navigation Jump to search
Overview For users For resource providers Infrastructure status Site-specific configuration Architecture




EGI Federated Cloud jOCCI APIs

Introduction

The wiki is intended for programmers who work in/with scientific communities and want to port or develop new tools and services (PaaS solutions, Orchestrators, etc.) for scientific user communities on top of the EGI Federated Cloud.

Before to start

This wiki assumes you have:

  • Obtained a trusted X.509 digital certificate
  • Subscribed the fedcloud.egi.eu VO. If you are member of another VO that has access to cloud resources, you may use that VO
  • Created a valid proxy certificate starting from your X.509 digital certificate

If you do NOT have a personal X.509 digital certificate, please follow the instructions reported in this documentation: how to get access the EGI Federated Cloud infrastructure.

The EGI FedCloud Clients image is a VM image based on Ubuntu 14.04 LTS which includes:

  1. voms clients with the fedcloud.egi.eu (and other cloud VOs) preconfigured,
  2. rOCCI-cli tool,
  3. additional libraries,
  4. Java SDK 7+,
  5. Maven compiler.

Requirements

The following prior knowledge/skills are required:

  • Java programming;
  • Basic Unix shell (SSH, bash scripting).

Source codes & Java Docs

jOCCI-api is a Java library implementing transport functions for rendered OCCI (Open Cloud Computing Interface) queries. It is built on top of jOCCI-core and currently provides HTTP transport functionality with set of authentication methods and basic requesting interface to easily communicate with OCCI servers. Official jOCCI APIs can be found on GitHub repositories and on Maven central.

The repository with Java exercises to use the EGI Federated Cloud infrastructure with jOCCI APIs can be found on github

Java Docs are available at:

  • egi-fctf.github.io/jOCCI-core/apidocs/index.html
  • egi-fctf.github.io/jOCCI-api/apidocs/index.html

Installing source codes

Download from github the repository with the source codes and instructions to interact with the EGI FedCloud infrastructure with jOCCI APIs

]$ git clone https://github.com/EGI-FCTF/di4r-training
Cloning into 'di4r-training'...
remote: Counting objects: 595, done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 595 (delta 8), reused 0 (delta 0), pack-reused 545
Receiving objects: 100% (595/595), 4.14 MiB | 1.25 MiB/s, done.
Resolving deltas: 100% (233/233), done.
]$ tree -L 1 di4r-training/
di4r-training/
|-- Api-VOMS-CANL
|-- LICENSE
|-- README.md
|-- checkIP
|-- jEGIAppDB
|-- jOCCI-attach-resources
|-- jOCCI-create-resources
|-- jOCCI-delete-resources
|-- jOCCI-describe-resources
|-- jOCCI-dump-model
|-- jOCCI-list-resources
`-- pyEGIAppDB
10 directories, 2 files


Querying the EGI Application Database

Introduction

In this exercise REST technology will be used to get the list of certified cloud providers from the EGI Application Database. The API is located at http://appdb-pi.egi.eu and it allows information retrieval and modification from third party applications without having to reside on the rich user interface of the AppDB portal.

  • The base URI of the REST API is http://appdb-pi.egi.eu/rest/1.0/
  • Requests must be followed by at least a resource name which may be followed by one or more sub-resource names separated by slashes, as in the examples given bellow:

Get the list of scientific applications from the EGI AppDB:

http://appdb-pi.egi.eu/rest/1.0/applications/

Get the list of available cloud providers:

http://appdb-pi.egi.eu/rest/1.0/va_providers/

Get info about the provider having ID=8253G0:

http://appdb-pi.egi.eu/rest/1.0/va_providers/8253G0

Objective

  • For each provider a list of available Virtual Appliances (VAs) and resource templates will be shown.
  • Choose one of the available provider publishing the Jupyter Notebook VA and identify the following IDs:
  1. The Cloud Provider Endpoint you want to use
  2. The Jupyter Notebook VA Image ID for the given provider
  3. The Resource Template ID the Jupyter Notebook VM should use (please use the smallest!)

Usage

  • Access the maven project
]$ cd di4r-training/jEGIAppDB/
  • Edit the source code in src/main/java/it/infn/ct/jEGIAppDB.java to use your preferred VO:
// Specify the EGI AppDB endpoint to query
public static String endpoint = "appdb.egi.eu";
// Specify the VO name 
public static String VO = "training.egi.eu"; 
  • Compile and package with maven:
]$ mvn compile && mvn package
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jEGIAppDB 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.520s
[INFO] Finished at: Mon Oct 03 16:15:04 CEST 2016
[INFO] Final Memory: 8M/30M
[INFO] ------------------------------------------------------------------------
  • Run your executable jar (you may redirect output to a file):
]$ java -jar target/jEGIAppDB-1.0-jar-with-dependencies.jar

 ~ Listing providers that have subscribed the [training.egi.eu] VO 

- BIFI [5073G0] 
	--> Sitename: Instituto de Biocomputacion y Fisica de Sistemas Complejos
	--> Endpoint: N/A
	--> Status: Certified
	--> URL: https://appdb.egi.eu/rest/1.0/va_providers/5073G0

 ~ Listing available resource(s) templates 

	resource_tpl#m1-xlarge
	resource_tpl#m1-tiny
	resource_tpl#m1-xlarge_cpu
	resource_tpl#m1-small_lhcb
	resource_tpl#m1-large
	resource_tpl#m1-small_cern
	resource_tpl#m1-small
	resource_tpl#m1-medium

 ~ Listing available Virtual Appliance(s) 

	 - Name = COMPSs-PMES  [v1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#13299e61-ee04-4200-a26e-187a92f2b4bd
	 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1786/

	 - Name = EGI Docker (Ubuntu 14.04) [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#c837c124-fc3b-48f4-ad8f-f91009e83608
	 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:1784/

	 - Name = UMD SL6 [v20150921]
	 - OCCI ID = http://schemas.openstack.org/template/os#9d9d3d60-5bee-4bed-9dea-93d5a9ebd2ff
	 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:1787/

	 - Name = transPLANT Base [v1.1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#494a2e1e-88d6-4d41-aa31-9b4ac7675ab5
	 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:1788/

	 - Name = Jupyter Notebook [v4.0.6-4]
	 - OCCI ID = http://schemas.openstack.org/template/os#95a46c41-6f88-4111-8823-566d734b6d17
	 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/

	 - Name = EGI FedCloud Clients [v2016.05.03]
	 - OCCI ID = http://schemas.openstack.org/template/os#3ed9edc5-05b3-42cd-9032-9490eadd49eb
	 - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2361/

	 - Name = Fractal Docker Ubuntu 14.04 [v20160330]
	 - OCCI ID = http://schemas.openstack.org/template/os#3e6dce0f-9dbd-4718-9f5b-27a4ebece5cf
	 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2359/

	 - Name = Chipster [v3.6.2]
	 - OCCI ID = http://schemas.openstack.org/template/os#afe5f4d9-6773-438f-8cad-4beb07c24a2a
	 - URI = https://appdb.egi.eu/store/vo/image/5057da7b-d3bb-502e-84d1-3484daf2607f:1785/

	 - Name = EGI Ubuntu 12.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#4f4c26ed-8c34-4b50-916b-c8279d8cdc9c
	 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1793/

	 - Name = MoinMoin wiki [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#1e108e1c-6ab8-42dc-bde2-a84fa84e8459
	 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1790/

	 - Name = Cassandra [v20-2.0.17]
	 - OCCI ID = http://schemas.openstack.org/template/os#44060b7b-4366-4bbb-a171-07352e2748eb
	 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:1832/

	 - Name = EGI Ubuntu 14.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#2be89672-9c13-4240-96c5-fe0119f72df2
	 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1782/

	 - Name = EGI CentOS 7 [v2016.05.09]
	 - OCCI ID = http://schemas.openstack.org/template/os#565d535a-43bc-499d-9063-a3a1977b5dc4
	 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2362/

	 - Name = EGI Centos 6 [v2016.04.28]
	 - OCCI ID = http://schemas.openstack.org/template/os#e809efaa-2827-4981-9375-031e0db91af9
	 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2360/

- BIFI [5706G0] 
	--> Sitename: Instituto de Biocomputacion y Fisica de Sistemas Complejos
	--> Endpoint: http://server4-eupt.unizar.es:8787/
	--> Status: Certified
	--> URL: https://appdb.egi.eu/rest/1.0/va_providers/5706G0

 ~ Listing available resource(s) templates 

	resource_tpl#m1-xlarge
	resource_tpl#m1-tiny
	resource_tpl#m1-xlarge_cpu
	resource_tpl#m1-small_lhcb
	resource_tpl#m1-large
	resource_tpl#m1-small_cern
	resource_tpl#m1-small
	resource_tpl#m1-medium

 ~ Listing available Virtual Appliance(s) 

	 - Name = COMPSs-PMES  [v1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#13299e61-ee04-4200-a26e-187a92f2b4bd
	 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1786/

	 - Name = EGI Docker (Ubuntu 14.04) [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#c837c124-fc3b-48f4-ad8f-f91009e83608
	 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:1784/

	 - Name = UMD SL6 [v20150921]
	 - OCCI ID = http://schemas.openstack.org/template/os#9d9d3d60-5bee-4bed-9dea-93d5a9ebd2ff
	 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:1787/

	 - Name = transPLANT Base [v1.1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#494a2e1e-88d6-4d41-aa31-9b4ac7675ab5
	 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:1788/

	 - Name = Jupyter Notebook [v4.0.6-4]
	 - OCCI ID = http://schemas.openstack.org/template/os#95a46c41-6f88-4111-8823-566d734b6d17
	 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/

	 - Name = EGI FedCloud Clients [v2016.05.03]
	 - OCCI ID = http://schemas.openstack.org/template/os#3ed9edc5-05b3-42cd-9032-9490eadd49eb
	 - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2361/

	 - Name = Fractal Docker Ubuntu 14.04 [v20160330]
	 - OCCI ID = http://schemas.openstack.org/template/os#3e6dce0f-9dbd-4718-9f5b-27a4ebece5cf
	 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2359/

	 - Name = Chipster [v3.6.2]
	 - OCCI ID = http://schemas.openstack.org/template/os#afe5f4d9-6773-438f-8cad-4beb07c24a2a
	 - URI = https://appdb.egi.eu/store/vo/image/5057da7b-d3bb-502e-84d1-3484daf2607f:1785/

	 - Name = EGI Ubuntu 12.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#4f4c26ed-8c34-4b50-916b-c8279d8cdc9c
	 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1793/

	 - Name = MoinMoin wiki [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#1e108e1c-6ab8-42dc-bde2-a84fa84e8459
	 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1790/

	 - Name = Cassandra [v20-2.0.17]
	 - OCCI ID = http://schemas.openstack.org/template/os#44060b7b-4366-4bbb-a171-07352e2748eb
	 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:1832/

	 - Name = EGI Ubuntu 14.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#2be89672-9c13-4240-96c5-fe0119f72df2
	 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1782/

	 - Name = EGI CentOS 7 [v2016.05.09]
	 - OCCI ID = http://schemas.openstack.org/template/os#565d535a-43bc-499d-9063-a3a1977b5dc4
	 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2362/

	 - Name = EGI Centos 6 [v2016.04.28]
	 - OCCI ID = http://schemas.openstack.org/template/os#e809efaa-2827-4981-9375-031e0db91af9
	 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2360/

- CESNET-MetaCloud [4454G0] 
	--> Sitename: CESNET MetaCloud (IaaS Cloud)
	--> Endpoint: https://carach5.ics.muni.cz:11443
	--> Status: Certified
	--> URL: https://appdb.egi.eu/rest/1.0/va_providers/4454G0

 ~ Listing available resource(s) templates 

	 resource_tpl#mem_large
	 resource_tpl#mem_small
	 resource_tpl#sym_mammoth
	 resource_tpl#extra_large
	 resource_tpl#small
	 resource_tpl#universe
	 resource_tpl#mammoth
	 resource_tpl#mem_medium
	 resource_tpl#medium
	 resource_tpl#mem_extra_large
	 resource_tpl#large
	 resource_tpl#goliath
	 resource_tpl#atlas

 ~ Listing available Virtual Appliance(s) 

	 - Name = COMPSs-PMES  [v1.4]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_pmes_14_ubuntu_15_04_x86_64_0001_fedcloud_warg_143
	 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1103/

	 - Name = EGI Docker (Ubuntu 14.04) [v2016.06.06]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_docker_ubuntu_server_14_04_lts_fedcloud_warg_146
	 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:2618/

	 - Name = UMD SL6 [v20150921]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_umd_sl6_fedcloud_warg_137
	 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:772/

	 - Name = transPLANT Base [v1.1.4]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_transplant_base_fedcloud_warg_141
	 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:924/

	 - Name = Jupyter Notebook [v4.0.6-4]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162
	 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/

	 - Name = EGI FedCloud Clients [v2016.09.06]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_egi_fedcloud_clients_fedcloud_warg_127
	 - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2885/

	 - Name = Fractal Docker Ubuntu 14.04 [v20160330]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_fractal_docker_ubuntu_14_04_x86_64_fedcloud_warg_144
	 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2025/

	 - Name = EGI Ubuntu 12.04 [v20160222]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_ubuntu_server_12_04_lts_fedcloud_warg_122
	 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1779/

	 - Name = MoinMoin wiki [v20160120]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_moinmoinwiki_fedcloud_warg_126
	 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1452/

	 - Name = Cassandra [v20-2.0.17-1]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_apache_cassandra_centos_6_fedcloud_warg_163
	 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:2396/

	 - Name = EGI Ubuntu 14.04 [v20160222]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_ubuntu_server_14_04_lts_fedcloud_warg_121
	 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1780/

	 - Name = EGI CentOS 7 [v2016.05.31]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_centos_7_fedcloud_warg_160
	 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2502/

	 - Name = EGI Centos 6 [v2016.04.28]
	 - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_centos_6_fedcloud_warg_123
	 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2177/

- CETA-GRID [5727G0] 
	--> Sitename: CETA-GRID, Resource Center CETA-CIEMAT
	--> Endpoint: https://controller.ceta-ciemat.es:8787/
	--> Status: Certified
	--> URL: https://appdb.egi.eu/rest/1.0/va_providers/5727G0

 ~ Listing available resource(s) templates 

	resource_tpl#m1-tiny
	resource_tpl#m1-common_ramplus6p
	resource_tpl#m1-large
	resource_tpl#m1-common_ramplus8p
	resource_tpl#m1-small
	resource_tpl#m1-medium
	resource_tpl#m1-common

 ~ Listing available Virtual Appliance(s) 

	 - Name = COMPSs-PMES  [v1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#e938c82b-661b-48cd-a359-692c4a3cf8e8
	 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1337/

	 - Name = EGI Docker (Ubuntu 14.04) [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#540be855-2438-4a84-a786-a0a904ed9e10
	 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:1449/

	 - Name = UMD SL6 [v20150921]
	 - OCCI ID = http://schemas.openstack.org/template/os#8ec2a043-1869-454d-baaa-49dc10fe4039
	 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:778/

	 - Name = transPLANT Base [v1.1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#ea843836-7fd4-4fc0-90c9-2256403bc452
	 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:1341/

	 - Name = EGI FedCloud Clients [v2016.05.03]
	 - OCCI ID = http://schemas.openstack.org/template/os#270a8e4c-0704-487d-bcde-7d0a4dcad5b4
	 - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2201/

	 - Name = Fractal Docker Ubuntu 14.04 [v20151202]
	 - OCCI ID = http://schemas.openstack.org/template/os#5d8b83bb-0388-4805-9242-1e27edbbf4da
	 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:1345/

	 - Name = Fractal Docker Ubuntu 14.04 [v20160330]
	 - OCCI ID = http://schemas.openstack.org/template/os#6dfdd077-e5b7-4643-9172-13ecf4ac8a4c
	 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2025/

	 - Name = Chipster [v3.6.2]
	 - OCCI ID = http://schemas.openstack.org/template/os#95476989-91a6-40b7-9ba2-4a0cef8f3454
	 - URI = https://appdb.egi.eu/store/vo/image/5057da7b-d3bb-502e-84d1-3484daf2607f:1336/

	 - Name = EGI Ubuntu 12.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#288145a4-208a-40e3-ac5a-6521d79d82f8
	 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1779/

	 - Name = EGI Ubuntu 12.04 [v20150903]
	 - OCCI ID = http://schemas.openstack.org/template/os#4dd50e47-be77-4342-9eb7-acaae5d6bc41
	 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:695/

	 - Name = MoinMoin wiki [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#24d16a1f-1cec-48eb-8574-871e01ee412e
	 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1452/

	 - Name = Cassandra [v20-2.0.17]
	 - OCCI ID = http://schemas.openstack.org/template/os#eedf5974-587e-48c1-801f-a00d122aa9f1
	 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:1832/

	 - Name = EGI Ubuntu 14.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#b60e4fa9-4192-4978-b0bb-154020d5fb20
	 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1780/

	 - Name = EGI Ubuntu 14.04 [v20150903]
	 - OCCI ID = http://schemas.openstack.org/template/os#951881b6-d102-43ab-9f4c-70f4ddbfde5b
	 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:736/

	 - Name = EGI CentOS 7 [v2016.05.09]
	 - OCCI ID = http://schemas.openstack.org/template/os#453f74ce-4bf3-4a06-812f-9409753659b3
	 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2278/

	 - Name = EGI Centos 6 [v2016.04.28]
	 - OCCI ID = http://schemas.openstack.org/template/os#6fc992fe-3750-4374-81e6-7602e1b1b67c
	 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2177/

- MK-04-FINKICLOUD [5586G0] 
	--> Sitename: Cloud infrastructure, Faculty of Computer Science and Engineering
	--> Endpoint: https://occi.nebula.finki.ukim.mk:443
	--> Status: Certified
	--> URL: https://appdb.egi.eu/rest/1.0/va_providers/5586G0

 ~ Listing available resource(s) templates 

	resource_tpl#extra_large
	resource_tpl#medium
	resource_tpl#small
	resource_tpl#large

 ~ Listing available Virtual Appliance(s) 

	 - Name = COMPSs-PMES  [v1.4]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_compss_pmes_training_198
	 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1104/

	 - Name = EGI Docker (Ubuntu 14.04) [v2016.06.06]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_docker_ubuntu_14_04_training_197
	 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:2618/

	 - Name = UMD SL6 [v20150921]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_umd_sl6_training_199
	 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:1109/

	 - Name = Jupyter Notebook [v4.0.6-4]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_jupyter_notebook_training_201
	 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/

	 - Name = Fractal Docker Ubuntu 14.04 [v20160330]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_fractal_docker_ubuntu_14_04_training_200
	 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2025/

	 - Name = EGI Ubuntu 12.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_ubuntu_12_04_training_171
	 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1779/

	 - Name = MoinMoin wiki [v20160120]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_wiki_ubuntu_14_04_training_168
	 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1452/

	 - Name = Cassandra [v20-2.0.17-1]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_cassandra_training_202
	 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:2396/

	 - Name = EGI Ubuntu 14.04 [v20160222]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_ubuntu_14_04_training_170
	 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1780/

	 - Name = EGI CentOS 7 [v2016.09.20]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_egi_centos_7_training_196
	 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2946/

	 - Name = EGI Centos 6 [v2016.09.20]
	 - OCCI ID = http://schemas.openstack.org/template/os#uuid_centos_6_training_172
	 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2945/

For CESNET-MetaCloud provider, we have the following IDs:


Create your Jupyter Notebook in the EGI FedCloud Infrastructure

Objective

In this exercise we will show how to use jOCCI APIs to:

  • Create your Jupyter Notebook in one of the providers ofthe EGI Federation
    • The contextualization process will be used to inject SSH public key and specify a script executed at boot time
    • cloud-init tool is used in the EGI Federation to implement contextualization
  • Get the IP address of your running notebook
  • Attach a public network IP (if necessary!)
  • Access the Jupyter Notebook via SSH/Web

Usage

  • Access the maven project
]$ cd di4r-traning/jOCCI-create-resources/
  • Edit your settings in the src/main/java/it/infn/ct/Exercise4.java source code to create a new compute resources:
// [ Setting preferences here! ]
public static String ACTION = "create";
public static String AUTH = "x509";

public static String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
public static String PROXY_PATH = "/tmp/x509up_u1000";

public static String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";

// *CREATE* a resources. Possible values are: 'compute' or 'storage'
public static List<String> RESOURCE = Arrays.asList("compute");

public static List<String> MIXIN = 
Arrays.asList("resource_tpl#medium",
"http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162");

public static List<String> CONTEXT =  
Arrays.asList("public_key=file:/home/ubuntu/.ssh/id_rsa.pub",
"user_data=file:/home/ubuntu/di4r-training/jOCCI-create-resources/contextualization.txt");

public static List<String> ATTRIBUTES = Arrays.asList("occi.core.title=Jupyter");
    
public static String OCCI_PUBLICKEY_NAME = "centos"; // <= Do NOT change it!
  • Compile and package with maven:
 ]$ mvn compile && mvn package
 [INFO] Scanning for projects...
 [INFO]                                                                         
 [INFO] ------------------------------------------------------------------------
 [INFO] Building jocci-create-resource 1.0
 [INFO] ------------------------------------------------------------------------
 [..]
 [INFO] ------------------------------------------------------------------------
 [INFO] BUILD SUCCESS
 [INFO] ------------------------------------------------------------------------
 [INFO] Total time: 11.120s
 [INFO] Finished at: Mon Oct 03 17:01:52 CEST 2016
 [INFO] Final Memory: 10M/30M
 [INFO] ------------------------------------------------------------------------
  • Start your Jupyter Notebook (you may redirect the output to a file):
]$ java -jar target/jOCCI-create-resource-1.0-jar-with-dependencies.jar

[ACTION] = create
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [compute]
MIXIN = [resource_tpl#medium, 
         http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162]

TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
CONTEXT = [public_key=file:/home/ubuntu/.ssh/id_rsa.pub, 
           user_data=file:/home/ubuntu/APIs/jOCCI-create-resources/contextualization.txt]

OCCI_PUBLICKEY_NAME = centos
ATTRIBUTES = [occi.core.title=Jupyter]
Verbose = True 

[+] Creating a new compute Virtual Machine (VM)
URI = https://carach5.ics.muni.cz:11443/compute/76623
  • Save your Jupyter Notebook VM ID

In this example, the Jupyter Notebook VM ID = https://carach5.ics.muni.cz:11443/compute/76623

Get Jupyter Notebook metadata

  • Access the maven project
]$ cd di4r-traning/jOCCI-describe-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise3.java source code
String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";
public static String AUTH = "x509";
public static String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
String PROXY_PATH = "/tmp/x509up_u1000";

String ACTION = "describe";
List<String> RESOURCE = Arrays.asList("compute", "https://carach5.ics.muni.cz:11443/compute/76623");
  • Compile and package with maven:
 ]$ mvn compile && mvn package
 [INFO] Scanning for projects...
 [INFO]                                                                         
 [INFO] ------------------------------------------------------------------------
 [INFO] Building jocci-describe-resources 1.0
 [INFO] ------------------------------------------------------------------------
 [..]
 [INFO] ------------------------------------------------------------------------
 [INFO] BUILD SUCCESS
 [INFO] ------------------------------------------------------------------------
 [INFO] Total time: 11.373s
 [INFO] Finished at: Mon Oct 03 17:17:18 CEST 2016
 [INFO] Final Memory: 10M/30M
 [INFO] ------------------------------------------------------------------------
  • Get metadata of your running Jupyter Notebook:
]$ java -jar target/jocci-describe-resources-1.0-jar-with-dependencies.jar 

[ACTION] = describe
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [compute, https://carach5.ics.muni.cz:11443/compute/76623]
TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
Verbose = ON 

[ VM DESCRIPTION ]
Category: compute;scheme="http://schemas.ogf.org/occi/infrastructure#";class="kind"
Category: medium;scheme="http://fedcloud.egi.eu/occi/compute/flavour/1.0#";class="mixin"
Category: uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162;scheme="
http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#";class="mixin"
Category: compute;scheme="http://opennebula.org/occi/infrastructure#";class="mixin"
Category: user_data;scheme="http://schemas.openstack.org/compute/instance#";class="mixin"
Category: public_key;scheme="http://schemas.openstack.org/instance/credentials#";class="mixin"
X-OCCI-Attribute: occi.compute.cores=2
X-OCCI-Attribute: occi.compute.memory=2.0
X-OCCI-Attribute: occi.compute.state="active"
X-OCCI-Attribute: occi.core.id=76623
X-OCCI-Attribute: occi.core.summary="Instantiated with rOCCI-server on Mon, 03 Oct 2016 17:02:26 +0200."
X-OCCI-Attribute: occi.core.title="Jupyter"
X-OCCI-Attribute: org.opennebula.compute.cpu=2.0
X-OCCI-Attribute: org.opennebula.compute.id=76623
[..]
Category: compute
scheme="http://schemas.ogf.org/occi/infrastructure#"
class="kind"
Category: medium
scheme="http://fedcloud.egi.eu/occi/compute/flavour/1.0#"
class="mixin"
Category: uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162
scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#"
class="mixin"
Category: compute
scheme="http://opennebula.org/occi/infrastructure#"
class="mixin"
Category: user_data
scheme="http://schemas.openstack.org/compute/instance#"
class="mixin"
Category: public_key
scheme="http://schemas.openstack.org/instance/credentials#"
class="mixin"
X-OCCI-Attribute: occi.compute.cores=2
X-OCCI-Attribute: occi.compute.memory=2.0
X-OCCI-Attribute: occi.compute.state="active"
X-OCCI-Attribute: occi.core.id=76623
X-OCCI-Attribute: occi.core.summary="Instantiated with rOCCI-server on Mon, 03 Oct 2016 17:02:26 +0200."
X-OCCI-Attribute: occi.core.title="Jupyter"
X-OCCI-Attribute: org.opennebula.compute.cpu=2.0
X-OCCI-Attribute: org.opennebula.compute.id=76623
X-OCCI-Attribute: org.openstack.compute.user_data="IyEvYmluL3NoCmVjaG8gIlRoaXM[..]hdGUgLVIpISIgPj4gL3RtcC9sb2cudHh0Cg=="
X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa AAAAB3NzaC1[..]hU3w84tp ubuntu@egi-fedcloud-clients"

Link: </network/24>
rel="http://schemas.ogf.org/occi/infrastructure#network"
self="/link/networkinterface/compute_76623_nic_0"
category="http://schemas.ogf.org/occi/infrastructure#networkinterface 
 http://opennebula.org/occi/infrastructure#networkinterface 
 http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface"
occi.core.id="compute_76623_nic_0"
occi.core.source="/compute/76623"
occi.core.target="/network/24"
occi.core.title="public"
occi.networkinterface.address="147.228.242.12"   
occi.networkinterface.interface="eth0"
occi.networkinterface.mac="02:00:93:e4:f2:0c"
occi.networkinterface.state="active"
org.opennebula.networkinterface.bridge="onebr0"

Link: </storage/3017>
rel="http://schemas.ogf.org/occi/infrastructure#storage"
self="/link/storagelink/compute_76623_disk_0"
category="http://schemas.ogf.org/occi/infrastructure#storagelink 
 http://opennebula.org/occi/infrastructure#storagelink"
occi.core.id="compute_76623_disk_0"
occi.core.source="/compute/76623"
occi.core.target="/storage/3017"
occi.core.title="20f1eac8-68b4-5938-b720-ab07b2edc485"
occi.storagelink.deviceid="/dev/vda"
occi.storagelink.state="active"

Link: </compute/76623?action=restart>
rel="http://schemas.ogf.org/occi/infrastructure/compute/action#restart"

Link: </compute/76623?action=stop>
rel="http://schemas.ogf.org/occi/infrastructure/compute/action#stop"

Link: </compute/76623?action=suspend>
rel="http://schemas.ogf.org/occi/infrastructure/compute/action#suspend"

According to this log, the Jupyter Notebook has been successfully started with the following network interface:

occi.core.title="public"
occi.networkinterface.address="147.228.242.12"   
occi.networkinterface.interface="eth0"
occi.networkinterface.mac="02:00:93:e4:f2:0c"

Since the Jupyter Notebook IP address (147.228.242.12) is public, we do NOT need to attach any public network to access the running VM via SSH (this is not always true!).

In case your VM is behind a NAT server you need to attach a public IP address using the jOCCI-attach-resources project.

For further details, check this wiki how to attach a public network to a running VM.

]$ ssh centos@147.228.242.12
The authenticity of host '147.228.242.12 (147.228.242.12)' can't be established.
RSA key fingerprint is 73:a0:ff:2e:5d:b4:a3:43:41:71:91:18:09:cd:f2:a2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '147.228.242.12' (RSA) to the list of known hosts.
Last login: Fri May 20 08:16:29 2016 from areagrid.ct.infn.it
[centos@stoor12 ~]$

Start the Jupyter Notebook as follows:

[centos@stoor12 ~]$ jupyter notebook
[W 17:35:21.117 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using authentication. 
This is highly insecure and not recommended.
[I 17:35:21.124 NotebookApp] Serving notebooks from local directory: /home/centos
[I 17:35:21.124 NotebookApp] 0 active kernels 
[I 17:35:21.124 NotebookApp] The Jupyter Notebook is running at: https://[all ip addresses on your system]:8888/
[I 17:35:21.124 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Now, the Jupyter Notebook can be accessed via web at: https://147.228.242.12:8888


Jupyter-2.png

Create a persistent storage

Objective

In this exercise we will show how to use jOCCI APIs to:

  • Create a block storage
  • Describe the information of a block storage
  • Attach the block storage to the Jupyter Notebook
  • Manage the block storage (e.g. mount and umount)

Usage

  • Access the maven project
]$ cd di4r-traning/jOCCI-create-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise4.java source code
public static String ACTION = "create";
public static String AUTH = "x509";
public static String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
public static String PROXY_PATH = "/tmp/x509up_u1000";

public static String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";

// *CREATE* a resources. Possible values are: 'compute' or 'storage'
public static List<String> RESOURCE = Arrays.asList("storage");

// *CREATE* a new block volume
public static List<String> ATTRIBUTES =
 Arrays.asList("occi.core.title=VM_volume_1", "occi.storage.size=1");

public static String OCCI_PUBLICKEY_NAME = "";
public static List<String> CONTEXT = new ArrayList<String>();
public static List<String> MIXIN = new ArrayList<String>();

public static Boolean verbose = true;
  • Compile and package with maven:
]$ mvn compile && mvn package
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jocci-create-resource 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.393s
[INFO] Finished at: Mon Oct 03 18:11:00 CEST 2016
[INFO] Final Memory: 10M/30M
[INFO] ------------------------------------------------------------------------
  • Create a persistent block storage:
]$ java -jar target/jocci-create-resource-1.0-jar-with-dependencies.jar 

[ACTION] = create
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [storage]
TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
ATTRIBUTES = [occi.core.title=VM_volume_1, occi.storage.size=1]
Verbose = True 

[+] Creating a volume storage
URI = https://carach5.ics.muni.cz:11443/storage/4263
  • Save the Block Storage ID

In this example, the Block Storage ID = https://carach5.ics.muni.cz:11443/storage/4263

Attach the block storage

  • Access the maven project
]$ cd di4r-traning/jOCCI-attach-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise6.java source code
String AUTH = "x509";
String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";
String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
String PROXY_PATH = "/tmp/x509up_u1000";
Boolean verbose = true;

List<String> RESOURCE = Arrays.asList("compute",  
"https://carach5.ics.muni.cz:11443/compute/76623");
String LINK_RESOURCE = ("https://carach5.ics.muni.cz:11443/storage/4263");
  • Compile and package with maven:
]$ mvn compile && mvn package
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jocci-attach-resources 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.491s
[INFO] Finished at: Mon Oct 03 18:20:21 CEST 2016
[INFO] Final Memory: 10M/30M
[INFO] ------------------------------------------------------------------------
  • Attach the block storage to your Jupyter Notebook
]$ java -jar target/jocci-attach-resources-1.0-jar-with-dependencies.jar 

[ACTION] = link
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [compute, https://carach5.ics.muni.cz:11443/compute/76623]
TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
LINK_RESOURCE = https://carach5.ics.muni.cz:11443/storage/4263
Verbose = True 

[+] Attach a volume to the VM
https://carach5.ics.muni.cz:11443/compute/76623
[-] VolumeID
https://carach5.ics.muni.cz:11443/storage/4263

Get Jupyter Notebook metadata

  • Access the maven project
]$ cd di4r-traning/jOCCI-describe-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise3.java source code
String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";
public static String AUTH = "x509";
public static String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
String PROXY_PATH = "/tmp/x509up_u1000";

String ACTION = "describe";
List<String> RESOURCE = Arrays.asList("compute", "https://carach5.ics.muni.cz:11443/compute/76623");
  • Compile and package with maven:
]$ mvn compile && mvn package
INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jocci-describe-resources 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.356s
[INFO] Finished at: Tue Oct 03 18:17:56 CEST 2016
[INFO] Final Memory: 10M/30M
[INFO] ------------------------------------------------------------------------
  • Get metadata of your running Jupyter Notebook:
]$ java -jar target/jocci-describe-resources-1.0-jar-with-dependencies.jar
[..]
Link: </storage/3017>
rel="http://schemas.ogf.org/occi/infrastructure#storage"
self="/link/storagelink/compute_76623_disk_0"
category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink"
occi.core.id="compute_76623_disk_0"
occi.core.source="/compute/76623"
occi.core.target="/storage/3017"
occi.core.title="20f1eac8-68b4-5938-b720-ab07b2edc485"
occi.storagelink.deviceid="/dev/vda"
occi.storagelink.state="active"

Link: </storage/4263>
rel="http://schemas.ogf.org/occi/infrastructure#storage"
self="/link/storagelink/compute_76623_disk_2"
category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink"
occi.core.id="compute_76623_disk_2"
occi.core.source="/compute/76623"
occi.core.target="/storage/4263"
occi.core.title="VM_volume_1"
occi.storagelink.deviceid="/dev/vdc"
occi.storagelink.state="active"

Link: </compute/76623?action=restart>
rel="http://schemas.ogf.org/occi/infrastructure/compute/action#restart"

Link: </compute/76623?action=stop>
rel="http://schemas.ogf.org/occi/infrastructure/compute/action#stop"

Link: </compute/76623?action=suspend>
rel="http://schemas.ogf.org/occi/infrastructure/compute/action#suspend"

According to this log, the persistent block storage is identified by the following IDs:

self="/link/storagelink/compute_76623_disk_2" <= Storage Link ID 
occi.storagelink.deviceid="/dev/vdc" <= Device to manage the volume

Mount block storage

  • Access the Jupyter Notebook
]$ ssh centos@147.228.242.12
  • Preparing the block storage before to use it
[centos@stoor12 ~]$ sudo mkfs.ext3 /dev/vdc 
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
  • Mount the block storage
[centos@stoor12 ~]$ sudo mount /dev/vdc  /mnt

[centos@stoor12 ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_jupytersmall-lv_root
                       18G   11G  6.0G  64% /
tmpfs                 940M   72K  939M   1% /dev/shm
/dev/vda1             477M   80M  372M  18% /boot
/dev/vdc             1008M   34M  924M   4% /mnt
  • Unmount the block storage
[centos@stoor12 ~]$ sudo umount /mnt


Clean up!

Objective

In this exercise we will show how to use jOCCI APIs to:

  • Detach and Delete allocated resources (e.g. compute, storage)

Usage

  • Access the maven project
]$ cd di4r-traning/jOCCI-delete-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise5.java source code
String AUTH = "x509";
String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
String PROXY_PATH = "/tmp/x509up_u1000";

String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";
String ACTION = "delete";

// - Deleting storage link
List<String> RESOURCE = Arrays.asList("storage",  "/link/storagelink/compute_76623_disk_2");

Boolean verbose = true;
  • Compile and package with maven:
]$ mvn compile && mvn package
INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jocci-delete-resources 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.356s
[INFO] Finished at: Tue Oct 03 18:23:56 CEST 2016
[INFO] Final Memory: 10M/30M
[INFO] ------------------------------------------------------------------------
  • Delete the storage link:
]$ java -jar target/jocci-delete-resources-1.0-jar-with-dependencies.jar 

[ACTION] = delete
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [storage, /link/storagelink/compute_76623_disk_2]
TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
Verbose = True 

[+] Clean up!
[-] Deleted: OK

Delete the Block Storage

  • Access the maven project
]$ cd di4r-traning/jOCCI-delete-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise5.java source code
String AUTH = "x509";
String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
String PROXY_PATH = "/tmp/x509up_u1000";

String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";
String ACTION = "delete";

// - Deleting block storage
List<String> RESOURCE = Arrays.asList("storage", "https://carach5.ics.muni.cz:11443/storage/4263");
  • Compile and package with maven:
]$ mvn compile && mvn package
INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jocci-delete-resources 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.356s
[INFO] Finished at: Tue Oct 03 18:30:56 CEST 2016
[INFO] Final Memory: 10M/30M
[INFO] ------------------------------------------------------------------------
  • Delete the storage link:
]$ java -jar target/jocci-delete-resources-1.0-jar-with-dependencies.jar 
[ACTION] = delete
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [storage, https://carach5.ics.muni.cz:11443/storage/4263]
TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
Verbose = True 

[+] Clean up!
[-] Deleted: OK

Delete the Jupyter Notebook

  • Access the maven project
]$ cd di4r-traning/jOCCI-delete-resources/
  • Configure your settings in the src/main/java/it/infn/ct/Exercise5.java source code
String AUTH = "x509";
String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates";
String PROXY_PATH = "/tmp/x509up_u1000";

String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443";
String ACTION = "delete";

// - Deleting block storage
List<String> RESOURCE = Arrays.asList("compute", "https://carach5.ics.muni.cz:11443/compute/76623");
  • Compile and package with maven:
]$ mvn compile && mvn package
INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building jocci-delete-resources 1.0
[INFO] ------------------------------------------------------------------------
[..]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.356s
[INFO] Finished at: Tue Oct 03 18:35:16 CEST 2016
[INFO] Final Memory: 10M/30M
[INFO] ------------------------------------------------------------------------
  • Delete the storage link:
]$ java -jar target/jocci-delete-resources-1.0-jar-with-dependencies.jar 
[ACTION] = delete
AUTH = x509
OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443
RESOURCE = [compute, https://carach5.ics.muni.cz:11443/compute/76623]
TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates
PROXY_PATH = /tmp/x509up_u1000
Verbose = True 

[+] Trigger a 'stop' action to the 'compute' resource
[-] Triggered: OK
[+] Delete the virtual appliance
[-] Deleted: OK