Difference between revisions of "EGI Federated Cloud jOCCI APIs"
Line 678: | Line 678: | ||
X-OCCI-Attribute: org.opennebula.compute.cpu=2.0 | X-OCCI-Attribute: org.opennebula.compute.cpu=2.0 | ||
X-OCCI-Attribute: org.opennebula.compute.id=76623 | X-OCCI-Attribute: org.opennebula.compute.id=76623 | ||
X-OCCI-Attribute: org.openstack.compute.user_data=" | X-OCCI-Attribute: org.openstack.compute.user_data="IyEvYmluL3NoCmVjaG8gIlRoaXMgaXMgYSBzaW1wbGUgQ[..]LVIpISIgPj4gL3RtcC9sb2cudHh0Cg==" | ||
X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa | X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAB[..]84tp ubuntu@egi-fedcloud-clients" | ||
Link: </network/24>;rel="http://schemas.ogf.org/occi/infrastructure#network";self="/link/networkinterface/compute_76623_nic_0";category="http://schemas.ogf.org/occi/infrastructure#networkinterface http://opennebula.org/occi/infrastructure#networkinterface http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface";occi.core.id="compute_76623_nic_0";occi.core.source="/compute/76623";occi.core.target="/network/24";occi.core.title="public";occi.networkinterface.address="147.228.242.12";occi.networkinterface.interface="eth0";occi.networkinterface.mac="02:00:93:e4:f2:0c";occi.networkinterface.state="active";org.opennebula.networkinterface.bridge="onebr0"; | Link: </network/24>;rel="http://schemas.ogf.org/occi/infrastructure#network";self="/link/networkinterface/compute_76623_nic_0";category="http://schemas.ogf.org/occi/infrastructure#networkinterface http://opennebula.org/occi/infrastructure#networkinterface http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface";occi.core.id="compute_76623_nic_0";occi.core.source="/compute/76623";occi.core.target="/network/24";occi.core.title="public";occi.networkinterface.address="147.228.242.12";occi.networkinterface.interface="eth0";occi.networkinterface.mac="02:00:93:e4:f2:0c";occi.networkinterface.state="active";org.opennebula.networkinterface.bridge="onebr0"; | ||
Link: </storage/3017>;rel="http://schemas.ogf.org/occi/infrastructure#storage";self="/link/storagelink/compute_76623_disk_0";category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink";occi.core.id="compute_76623_disk_0";occi.core.source="/compute/76623";occi.core.target="/storage/3017";occi.core.title="20f1eac8-68b4-5938-b720-ab07b2edc485";occi.storagelink.deviceid="/dev/vda";occi.storagelink.state="active"; | Link: </storage/3017>;rel="http://schemas.ogf.org/occi/infrastructure#storage";self="/link/storagelink/compute_76623_disk_0";category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink";occi.core.id="compute_76623_disk_0";occi.core.source="/compute/76623";occi.core.target="/storage/3017";occi.core.title="20f1eac8-68b4-5938-b720-ab07b2edc485";occi.storagelink.deviceid="/dev/vda";occi.storagelink.state="active"; |
Revision as of 17:20, 3 October 2016
Overview | For users | For resource providers | Infrastructure status | Site-specific configuration | Architecture |
EGI Federated Cloud jOCCI APIs
Introduction
The wiki is intended for programmers who work in/with scientific communities and want to port or develop new tools and services (PaaS solutions, Orchestrators, etc.) for scientific user communities on top of the EGI Federated Cloud.
Before to start
This wiki assumes that you have:
- Obtained a trusted X.509 digital certificate,
- Subscribed the fedcloud.egi.eu VO. If you are member of another VO that has access to cloud resources, you may use that VO
- Created a valid proxy certificate starting from your X.509 digital certificate
If you do NOT have a personal X.509 digital certificate, please follow the instructions reported in this documentation: how to get access the EGI Federated Cloud infrastructure.
- Access to the EGI FedCloud Clients Virtual Appliance
The EGI FedCloud Clients image is a VM image based on Ubuntu 14.04 LTS which includes:
- voms clients with the fedcloud.egi.eu (and other cloud VOs) preconfigured,
- rOCCI-cli tool,
- additional libraries,
- Java SDK 7+,
- Maven compiler.
Source codes & Java Docs
jOCCI-api is a Java library implementing transport functions for rendered OCCI (Open Cloud Computing Interface) queries. It is built on top of jOCCI-core and currently provides HTTP transport functionality with set of authentication methods and basic requesting interface to easily communicate with OCCI servers. Official jOCCI APIs can be found on GitHub repositories and on Maven central.
The repository with Java exercises to use the EGI Federated Cloud infrastructure with jOCCI APIs can be found on github
Java Docs are available at:
- egi-fctf.github.io/jOCCI-core/apidocs/index.html
- egi-fctf.github.io/jOCCI-api/apidocs/index.html
Installing source codes
Download from github the repository with the source codes and instructions to interact with the EGI FedCloud infrastructure with jOCCI APIs
]$ git clone https://github.com/EGI-FCTF/di4r-training Cloning into 'di4r-training'... remote: Counting objects: 595, done. remote: Compressing objects: 100% (20/20), done. remote: Total 595 (delta 8), reused 0 (delta 0), pack-reused 545 Receiving objects: 100% (595/595), 4.14 MiB | 1.25 MiB/s, done. Resolving deltas: 100% (233/233), done.
]$ tree -L 1 di4r-training/ di4r-training/ |-- Api-VOMS-CANL |-- LICENSE |-- README.md |-- checkIP |-- jEGIAppDB |-- jOCCI-attach-resources |-- jOCCI-create-resources |-- jOCCI-delete-resources |-- jOCCI-describe-resources |-- jOCCI-dump-model |-- jOCCI-list-resources `-- pyEGIAppDB 10 directories, 2 files
Exercise 1 - Querying the EGI Application Database
Introduction
In this exercise REST technology will be used to get the list of certified cloud providers from the EGI Application Database. The API is located at http://appdb-pi.egi.eu and it allows information retrieval and modification from third party applications without having to reside on the rich user interface of the AppDB portal.
- The base URI of the REST API is http://appdb-pi.egi.eu/rest/1.0/
- Requests must be followed by at least a resource name which may be followed by one or more sub-resource names separated by slashes, as in the examples given bellow:
- Get the list of scientific applications from the EGI AppDB:
- http://appdb-pi.egi.eu/rest/1.0/applications/
- Get the list of available cloud providers:
- http://appdb-pi.egi.eu/rest/1.0/va_providers/
- Get info about the provider having ID=8253G0:
- http://appdb-pi.egi.eu/rest/1.0/va_providers/8253G0
Objective
- For each provider a list of available Virtual Appliances (VAs) and resource templates will be shown.
- Choose one of the available provider publishing the Jupyter Notebook VA and identify the following IDs:
- The Cloud Provider Endpoint you want to use
- The Jupyter Notebook VA Image ID for the given provider
- The Resource Template ID the Jupyter Notebook VM should use (please use the smallest!)
Usage
- Access the maven project
]$ cd di4r-training/jEGIAppDB/
- Edit the source code in src/main/java/it/infn/ct/jEGIAppDB.java to use your preferred VO:
// Specify the EGI AppDB endpoint to query public static String endpoint = "appdb.egi.eu";
// Specify the VO name public static String VO = "training.egi.eu";
- Compile and package with maven:
]$ mvn compile && mvn package [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building jEGIAppDB 1.0 [INFO] ------------------------------------------------------------------------ [WARNING] The artifact xml-apis:xml-apis:jar:2.0.2 has been relocated to xml-apis:xml-apis:jar:1.0.b2 [INFO] [..] [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] META-INF/maven/com.sun.jersey/ already added, skipping [INFO] com/ already added, skipping [INFO] com/sun/ already added, skipping [INFO] com/sun/jersey/ already added, skipping [INFO] com/sun/jersey/api/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] javax/ already added, skipping [INFO] Building jar: /home/ubuntu/APIs/jEGIAppDB/target/jEGIAppDB-1.0-jar-with-dependencies.jar [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/maven/ already added, skipping [INFO] META-INF/maven/com.sun.jersey/ already added, skipping [INFO] com/ already added, skipping [INFO] com/sun/ already added, skipping [INFO] com/sun/jersey/ already added, skipping [INFO] com/sun/jersey/api/ already added, skipping [INFO] META-INF/ already added, skipping [INFO] META-INF/MANIFEST.MF already added, skipping [INFO] javax/ already added, skipping [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.520s [INFO] Finished at: Mon Oct 03 16:15:04 CEST 2016 [INFO] Final Memory: 8M/30M [INFO] ------------------------------------------------------------------------
- Run your executable jar (you may redirect output to a file):
]$ java -jar target/jEGIAppDB-1.0-jar-with-dependencies.jar ~ Listing providers that have subscribed the [training.egi.eu] VO - BIFI [5073G0] --> Sitename: Instituto de Biocomputacion y Fisica de Sistemas Complejos --> Endpoint: N/A --> Status: Certified --> URL: https://appdb.egi.eu/rest/1.0/va_providers/5073G0 ~ Listing available resource(s) templates resource_tpl#m1-xlarge resource_tpl#m1-tiny resource_tpl#m1-xlarge_cpu resource_tpl#m1-small_lhcb resource_tpl#m1-large resource_tpl#m1-small_cern resource_tpl#m1-small resource_tpl#m1-medium ~ Listing available Virtual Appliance(s) - Name = COMPSs-PMES [v1.4] - OCCI ID = http://schemas.openstack.org/template/os#13299e61-ee04-4200-a26e-187a92f2b4bd - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1786/ - Name = EGI Docker (Ubuntu 14.04) [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#c837c124-fc3b-48f4-ad8f-f91009e83608 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:1784/ - Name = UMD SL6 [v20150921] - OCCI ID = http://schemas.openstack.org/template/os#9d9d3d60-5bee-4bed-9dea-93d5a9ebd2ff - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:1787/ - Name = transPLANT Base [v1.1.4] - OCCI ID = http://schemas.openstack.org/template/os#494a2e1e-88d6-4d41-aa31-9b4ac7675ab5 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:1788/ - Name = Jupyter Notebook [v4.0.6-4] - OCCI ID = http://schemas.openstack.org/template/os#95a46c41-6f88-4111-8823-566d734b6d17 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/ - Name = EGI FedCloud Clients [v2016.05.03] - OCCI ID = http://schemas.openstack.org/template/os#3ed9edc5-05b3-42cd-9032-9490eadd49eb - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2361/ - Name = Fractal Docker Ubuntu 14.04 [v20160330] - OCCI ID = http://schemas.openstack.org/template/os#3e6dce0f-9dbd-4718-9f5b-27a4ebece5cf - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2359/ - Name = Chipster [v3.6.2] - OCCI ID = http://schemas.openstack.org/template/os#afe5f4d9-6773-438f-8cad-4beb07c24a2a - URI = https://appdb.egi.eu/store/vo/image/5057da7b-d3bb-502e-84d1-3484daf2607f:1785/ - Name = EGI Ubuntu 12.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#4f4c26ed-8c34-4b50-916b-c8279d8cdc9c - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1793/ - Name = MoinMoin wiki [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#1e108e1c-6ab8-42dc-bde2-a84fa84e8459 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1790/ - Name = Cassandra [v20-2.0.17] - OCCI ID = http://schemas.openstack.org/template/os#44060b7b-4366-4bbb-a171-07352e2748eb - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:1832/ - Name = EGI Ubuntu 14.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#2be89672-9c13-4240-96c5-fe0119f72df2 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1782/ - Name = EGI CentOS 7 [v2016.05.09] - OCCI ID = http://schemas.openstack.org/template/os#565d535a-43bc-499d-9063-a3a1977b5dc4 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2362/ - Name = EGI Centos 6 [v2016.04.28] - OCCI ID = http://schemas.openstack.org/template/os#e809efaa-2827-4981-9375-031e0db91af9 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2360/ - BIFI [5706G0] --> Sitename: Instituto de Biocomputacion y Fisica de Sistemas Complejos --> Endpoint: http://server4-eupt.unizar.es:8787/ --> Status: Certified --> URL: https://appdb.egi.eu/rest/1.0/va_providers/5706G0 ~ Listing available resource(s) templates resource_tpl#m1-xlarge resource_tpl#m1-tiny resource_tpl#m1-xlarge_cpu resource_tpl#m1-small_lhcb resource_tpl#m1-large resource_tpl#m1-small_cern resource_tpl#m1-small resource_tpl#m1-medium ~ Listing available Virtual Appliance(s) - Name = COMPSs-PMES [v1.4] - OCCI ID = http://schemas.openstack.org/template/os#13299e61-ee04-4200-a26e-187a92f2b4bd - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1786/ - Name = EGI Docker (Ubuntu 14.04) [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#c837c124-fc3b-48f4-ad8f-f91009e83608 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:1784/ - Name = UMD SL6 [v20150921] - OCCI ID = http://schemas.openstack.org/template/os#9d9d3d60-5bee-4bed-9dea-93d5a9ebd2ff - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:1787/ - Name = transPLANT Base [v1.1.4] - OCCI ID = http://schemas.openstack.org/template/os#494a2e1e-88d6-4d41-aa31-9b4ac7675ab5 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:1788/ - Name = Jupyter Notebook [v4.0.6-4] - OCCI ID = http://schemas.openstack.org/template/os#95a46c41-6f88-4111-8823-566d734b6d17 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/ - Name = EGI FedCloud Clients [v2016.05.03] - OCCI ID = http://schemas.openstack.org/template/os#3ed9edc5-05b3-42cd-9032-9490eadd49eb - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2361/ - Name = Fractal Docker Ubuntu 14.04 [v20160330] - OCCI ID = http://schemas.openstack.org/template/os#3e6dce0f-9dbd-4718-9f5b-27a4ebece5cf - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2359/ - Name = Chipster [v3.6.2] - OCCI ID = http://schemas.openstack.org/template/os#afe5f4d9-6773-438f-8cad-4beb07c24a2a - URI = https://appdb.egi.eu/store/vo/image/5057da7b-d3bb-502e-84d1-3484daf2607f:1785/ - Name = EGI Ubuntu 12.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#4f4c26ed-8c34-4b50-916b-c8279d8cdc9c - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1793/ - Name = MoinMoin wiki [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#1e108e1c-6ab8-42dc-bde2-a84fa84e8459 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1790/ - Name = Cassandra [v20-2.0.17] - OCCI ID = http://schemas.openstack.org/template/os#44060b7b-4366-4bbb-a171-07352e2748eb - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:1832/ - Name = EGI Ubuntu 14.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#2be89672-9c13-4240-96c5-fe0119f72df2 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1782/ - Name = EGI CentOS 7 [v2016.05.09] - OCCI ID = http://schemas.openstack.org/template/os#565d535a-43bc-499d-9063-a3a1977b5dc4 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2362/ - Name = EGI Centos 6 [v2016.04.28] - OCCI ID = http://schemas.openstack.org/template/os#e809efaa-2827-4981-9375-031e0db91af9 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2360/ - CESNET-MetaCloud [4454G0] --> Sitename: CESNET MetaCloud (IaaS Cloud) --> Endpoint: https://carach5.ics.muni.cz:11443 --> Status: Certified --> URL: https://appdb.egi.eu/rest/1.0/va_providers/4454G0 ~ Listing available resource(s) templates resource_tpl#mem_large resource_tpl#mem_small resource_tpl#sym_mammoth resource_tpl#extra_large resource_tpl#small resource_tpl#universe resource_tpl#mammoth resource_tpl#mem_medium resource_tpl#medium resource_tpl#mem_extra_large resource_tpl#large resource_tpl#goliath resource_tpl#atlas ~ Listing available Virtual Appliance(s) - Name = COMPSs-PMES [v1.4] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_pmes_14_ubuntu_15_04_x86_64_0001_fedcloud_warg_143 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1103/ - Name = EGI Docker (Ubuntu 14.04) [v2016.06.06] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_docker_ubuntu_server_14_04_lts_fedcloud_warg_146 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:2618/ - Name = UMD SL6 [v20150921] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_umd_sl6_fedcloud_warg_137 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:772/ - Name = transPLANT Base [v1.1.4] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_transplant_base_fedcloud_warg_141 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:924/ - Name = Jupyter Notebook [v4.0.6-4] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/ - Name = EGI FedCloud Clients [v2016.09.06] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_egi_fedcloud_clients_fedcloud_warg_127 - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2885/ - Name = Fractal Docker Ubuntu 14.04 [v20160330] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_fractal_docker_ubuntu_14_04_x86_64_fedcloud_warg_144 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2025/ - Name = EGI Ubuntu 12.04 [v20160222] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_ubuntu_server_12_04_lts_fedcloud_warg_122 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1779/ - Name = MoinMoin wiki [v20160120] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_moinmoinwiki_fedcloud_warg_126 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1452/ - Name = Cassandra [v20-2.0.17-1] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_apache_cassandra_centos_6_fedcloud_warg_163 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:2396/ - Name = EGI Ubuntu 14.04 [v20160222] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_ubuntu_server_14_04_lts_fedcloud_warg_121 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1780/ - Name = EGI CentOS 7 [v2016.05.31] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_centos_7_fedcloud_warg_160 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2502/ - Name = EGI Centos 6 [v2016.04.28] - OCCI ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_centos_6_fedcloud_warg_123 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2177/ - CETA-GRID [5727G0] --> Sitename: CETA-GRID, Resource Center CETA-CIEMAT --> Endpoint: https://controller.ceta-ciemat.es:8787/ --> Status: Certified --> URL: https://appdb.egi.eu/rest/1.0/va_providers/5727G0 ~ Listing available resource(s) templates resource_tpl#m1-tiny resource_tpl#m1-common_ramplus6p resource_tpl#m1-large resource_tpl#m1-common_ramplus8p resource_tpl#m1-small resource_tpl#m1-medium resource_tpl#m1-common ~ Listing available Virtual Appliance(s) - Name = COMPSs-PMES [v1.4] - OCCI ID = http://schemas.openstack.org/template/os#e938c82b-661b-48cd-a359-692c4a3cf8e8 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1337/ - Name = EGI Docker (Ubuntu 14.04) [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#540be855-2438-4a84-a786-a0a904ed9e10 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:1449/ - Name = UMD SL6 [v20150921] - OCCI ID = http://schemas.openstack.org/template/os#8ec2a043-1869-454d-baaa-49dc10fe4039 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:778/ - Name = transPLANT Base [v1.1.4] - OCCI ID = http://schemas.openstack.org/template/os#ea843836-7fd4-4fc0-90c9-2256403bc452 - URI = https://appdb.egi.eu/store/vo/image/1849f46a-5917-573d-bc4f-68058ef6dbd8:1341/ - Name = EGI FedCloud Clients [v2016.05.03] - OCCI ID = http://schemas.openstack.org/template/os#270a8e4c-0704-487d-bcde-7d0a4dcad5b4 - URI = https://appdb.egi.eu/store/vo/image/218cd70f-3058-5c9c-b3d0-50db8758e15f:2201/ - Name = Fractal Docker Ubuntu 14.04 [v20151202] - OCCI ID = http://schemas.openstack.org/template/os#5d8b83bb-0388-4805-9242-1e27edbbf4da - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:1345/ - Name = Fractal Docker Ubuntu 14.04 [v20160330] - OCCI ID = http://schemas.openstack.org/template/os#6dfdd077-e5b7-4643-9172-13ecf4ac8a4c - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2025/ - Name = Chipster [v3.6.2] - OCCI ID = http://schemas.openstack.org/template/os#95476989-91a6-40b7-9ba2-4a0cef8f3454 - URI = https://appdb.egi.eu/store/vo/image/5057da7b-d3bb-502e-84d1-3484daf2607f:1336/ - Name = EGI Ubuntu 12.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#288145a4-208a-40e3-ac5a-6521d79d82f8 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1779/ - Name = EGI Ubuntu 12.04 [v20150903] - OCCI ID = http://schemas.openstack.org/template/os#4dd50e47-be77-4342-9eb7-acaae5d6bc41 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:695/ - Name = MoinMoin wiki [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#24d16a1f-1cec-48eb-8574-871e01ee412e - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1452/ - Name = Cassandra [v20-2.0.17] - OCCI ID = http://schemas.openstack.org/template/os#eedf5974-587e-48c1-801f-a00d122aa9f1 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:1832/ - Name = EGI Ubuntu 14.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#b60e4fa9-4192-4978-b0bb-154020d5fb20 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1780/ - Name = EGI Ubuntu 14.04 [v20150903] - OCCI ID = http://schemas.openstack.org/template/os#951881b6-d102-43ab-9f4c-70f4ddbfde5b - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:736/ - Name = EGI CentOS 7 [v2016.05.09] - OCCI ID = http://schemas.openstack.org/template/os#453f74ce-4bf3-4a06-812f-9409753659b3 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2278/ - Name = EGI Centos 6 [v2016.04.28] - OCCI ID = http://schemas.openstack.org/template/os#6fc992fe-3750-4374-81e6-7602e1b1b67c - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2177/ - MK-04-FINKICLOUD [5586G0] --> Sitename: Cloud infrastructure, Faculty of Computer Science and Engineering --> Endpoint: https://occi.nebula.finki.ukim.mk:443 --> Status: Certified --> URL: https://appdb.egi.eu/rest/1.0/va_providers/5586G0 ~ Listing available resource(s) templates resource_tpl#extra_large resource_tpl#medium resource_tpl#small resource_tpl#large ~ Listing available Virtual Appliance(s) - Name = COMPSs-PMES [v1.4] - OCCI ID = http://schemas.openstack.org/template/os#uuid_compss_pmes_training_198 - URI = https://appdb.egi.eu/store/vo/image/02a34f93-1054-5441-816a-714eb2071118:1104/ - Name = EGI Docker (Ubuntu 14.04) [v2016.06.06] - OCCI ID = http://schemas.openstack.org/template/os#uuid_docker_ubuntu_14_04_training_197 - URI = https://appdb.egi.eu/store/vo/image/09232b7f-1a93-558e-b13c-7175b0fd4af4:2618/ - Name = UMD SL6 [v20150921] - OCCI ID = http://schemas.openstack.org/template/os#uuid_umd_sl6_training_199 - URI = https://appdb.egi.eu/store/vo/image/115aa4f1-fdd4-5129-bc19-09755e483940:1109/ - Name = Jupyter Notebook [v4.0.6-4] - OCCI ID = http://schemas.openstack.org/template/os#uuid_jupyter_notebook_training_201 - URI = https://appdb.egi.eu/store/vo/image/20f1eac8-68b4-5938-b720-ab07b2edc485:2381/ - Name = Fractal Docker Ubuntu 14.04 [v20160330] - OCCI ID = http://schemas.openstack.org/template/os#uuid_fractal_docker_ubuntu_14_04_training_200 - URI = https://appdb.egi.eu/store/vo/image/4151dc6f-5854-5bc0-a845-8dcc8b2920e3:2025/ - Name = EGI Ubuntu 12.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#uuid_ubuntu_12_04_training_171 - URI = https://appdb.egi.eu/store/vo/image/81be1590-2e18-58ca-a93c-d8c429aa081c:1779/ - Name = MoinMoin wiki [v20160120] - OCCI ID = http://schemas.openstack.org/template/os#uuid_wiki_ubuntu_14_04_training_168 - URI = https://appdb.egi.eu/store/vo/image/bcad2125-e9a3-5933-b415-55d440967481:1452/ - Name = Cassandra [v20-2.0.17-1] - OCCI ID = http://schemas.openstack.org/template/os#uuid_cassandra_training_202 - URI = https://appdb.egi.eu/store/vo/image/e2c97984-efdc-5f52-8c46-2f879619b6a2:2396/ - Name = EGI Ubuntu 14.04 [v20160222] - OCCI ID = http://schemas.openstack.org/template/os#uuid_ubuntu_14_04_training_170 - URI = https://appdb.egi.eu/store/vo/image/e72b7551-f7a4-523b-8832-da8725e49fc3:1780/ - Name = EGI CentOS 7 [v2016.09.20] - OCCI ID = http://schemas.openstack.org/template/os#uuid_egi_centos_7_training_196 - URI = https://appdb.egi.eu/store/vo/image/ecf1579e-cbee-5462-8b52-da953a82369f:2946/ - Name = EGI Centos 6 [v2016.09.20] - OCCI ID = http://schemas.openstack.org/template/os#uuid_centos_6_training_172 - URI = https://appdb.egi.eu/store/vo/image/ef631811-62b9-526c-8dce-c90cba3949b5:2945/
For CESNET-MetaCloud provider, we have the following IDs:
- Cloud Provider Endpoint = https://carach5.ics.muni.cz:11443
- Resource Template ID = resource_tpl#small
- VA Image ID = http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162
Exercise 2 - Start your Jupyter Notebook in the EGI FedCloud Infrastructure
Objective
In this exercise we will show how to use jOCCI APIs to:
- Create your Jupyter Notebook in one of the providers ofthe EGI Federation
- The contextualization process will be used to inject SSH public key and specify a script executed at boot time
- cloud-init tool is used in the EGI Federation to implement contextualization
- Get the IP address of your running notebook
- Attach a public network IP (if necessary!)
- Access the Jupyter Notebook via SSH/Web
Usage
- Access the maven project
]$ cd di4r-traning/jOCCI-create-resources/
- Edit your settings in the src/main/java/it/infn/ct/Exercise4.java source code to create a new compute resources:
// [ Setting preferences here! ] public static String ACTION = "create"; public static String AUTH = "x509"; public static String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates"; public static String PROXY_PATH = "/tmp/x509up_u1000"; public static String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443"; // *CREATE* a resources. Possible values are: 'compute' or 'storage' public static List<String> RESOURCE = Arrays.asList("compute"); public static List<String> MIXIN = Arrays.asList("resource_tpl#medium", "http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162"); public static List<String> CONTEXT = Arrays.asList("public_key=file:/home/ubuntu/.ssh/id_rsa.pub", "user_data=file:/home/ubuntu/di4r-training/jOCCI-create-resources/contextualization.txt"); public static List<String> ATTRIBUTES = Arrays.asList("occi.core.title=Jupyter"); public static String OCCI_PUBLICKEY_NAME = "centos"; // <= Do NOT change it!
- Compile and package with maven:
]$ mvn compile && mvn package [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building jocci-create-resource 1.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ jocci-create-resource --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /home/ubuntu/APIs/jOCCI-create-resources/src/main/resources [..] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 11.120s [INFO] Finished at: Mon Oct 03 17:01:52 CEST 2016 [INFO] Final Memory: 10M/30M [INFO] ------------------------------------------------------------------------
- Start your Jupyter Notebook (you may redirect the output to a file):
]$ java -jar target/jOCCI-create-resource-1.0-jar-with-dependencies.jar [ACTION] = create AUTH = x509 OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443 RESOURCE = [compute] MIXIN = [resource_tpl#medium, http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162] TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates PROXY_PATH = /tmp/x509up_u1000 CONTEXT = [public_key=file:/home/ubuntu/.ssh/id_rsa.pub, user_data=file:/home/ubuntu/APIs/jOCCI-create-resources/contextualization.txt] OCCI_PUBLICKEY_NAME = centos ATTRIBUTES = [occi.core.title=Jupyter] Verbose = True [+] Creating a new compute Virtual Machine (VM) URI = https://carach5.ics.muni.cz:11443/compute/76623
- Save your Jupyter Notebook VM ID
In this example, the Jupyter Notebook VM ID = https://carach5.ics.muni.cz:11443/compute/76623
Get Jupyter Notebook metadata
- Access the maven project
]$ cd di4r-traning/jOCCI-describe-resources/
- Configure your settings in the src/main/java/it/infn/ct/Exercise3.java source code
String OCCI_ENDPOINT_HOST = "https://carach5.ics.muni.cz:11443"; public static String AUTH = "x509"; public static String TRUSTED_CERT_REPOSITORY_PATH = "/etc/grid-security/certificates"; String PROXY_PATH = "/tmp/x509up_u1000"; String ACTION = "describe"; List<String> RESOURCE = Arrays.asList("compute", "https://carach5.ics.muni.cz:11443/compute/76623");
- Compile and package with maven:
]$ mvn compile && mvn package [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building jocci-describe-resources 1.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ jocci-describe-resources --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /home/ubuntu/APIs/jOCCI-describe-resources/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:2.0.2:compile (default-compile) @ jocci-describe-resources --- [..] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 11.373s [INFO] Finished at: Mon Oct 03 17:17:18 CEST 2016 [INFO] Final Memory: 10M/30M [INFO] ------------------------------------------------------------------------
- Get metadata of your running Jupyter Notebook:
$ java -jar target/jocci-describe-resources-1.0-jar-with-dependencies.jar [ACTION] = describe AUTH = x509 OCCI_ENDPOINT_HOST = https://carach5.ics.muni.cz:11443 RESOURCE = [compute, https://carach5.ics.muni.cz:11443/compute/76623] TRUSTED_CERT_REPOSITORY_PATH = /etc/grid-security/certificates PROXY_PATH = /tmp/x509up_u1000 Verbose = ON [ VM DESCRIPTION ] Category: compute;scheme="http://schemas.ogf.org/occi/infrastructure#";class="kind" Category: medium;scheme="http://fedcloud.egi.eu/occi/compute/flavour/1.0#";class="mixin" Category: uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162;scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#";class="mixin" Category: compute;scheme="http://opennebula.org/occi/infrastructure#";class="mixin" Category: user_data;scheme="http://schemas.openstack.org/compute/instance#";class="mixin" Category: public_key;scheme="http://schemas.openstack.org/instance/credentials#";class="mixin" X-OCCI-Attribute: occi.compute.cores=2 X-OCCI-Attribute: occi.compute.memory=2.0 X-OCCI-Attribute: occi.compute.state="active" X-OCCI-Attribute: occi.core.id=76623 X-OCCI-Attribute: occi.core.summary="Instantiated with rOCCI-server on Mon, 03 Oct 2016 17:02:26 +0200." X-OCCI-Attribute: occi.core.title="Jupyter" X-OCCI-Attribute: org.opennebula.compute.cpu=2.0 X-OCCI-Attribute: org.opennebula.compute.id=76623 X-OCCI-Attribute: org.openstack.compute.user_data="IyEvYmluL3NoCmVjaG8gIlRoaXMgaXMgYSBzaW1wbGUgQ[..]LVIpISIgPj4gL3RtcC9sb2cudHh0Cg==" X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAB[..]84tp ubuntu@egi-fedcloud-clients" Link: </network/24>;rel="http://schemas.ogf.org/occi/infrastructure#network";self="/link/networkinterface/compute_76623_nic_0";category="http://schemas.ogf.org/occi/infrastructure#networkinterface http://opennebula.org/occi/infrastructure#networkinterface http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface";occi.core.id="compute_76623_nic_0";occi.core.source="/compute/76623";occi.core.target="/network/24";occi.core.title="public";occi.networkinterface.address="147.228.242.12";occi.networkinterface.interface="eth0";occi.networkinterface.mac="02:00:93:e4:f2:0c";occi.networkinterface.state="active";org.opennebula.networkinterface.bridge="onebr0"; Link: </storage/3017>;rel="http://schemas.ogf.org/occi/infrastructure#storage";self="/link/storagelink/compute_76623_disk_0";category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink";occi.core.id="compute_76623_disk_0";occi.core.source="/compute/76623";occi.core.target="/storage/3017";occi.core.title="20f1eac8-68b4-5938-b720-ab07b2edc485";occi.storagelink.deviceid="/dev/vda";occi.storagelink.state="active"; Link: </compute/76623?action=restart>;rel="http://schemas.ogf.org/occi/infrastructure/compute/action#restart"; Link: </compute/76623?action=stop>;rel="http://schemas.ogf.org/occi/infrastructure/compute/action#stop"; Link: </compute/76623?action=suspend>;rel="http://schemas.ogf.org/occi/infrastructure/compute/action#suspend"; Category: compute scheme="http://schemas.ogf.org/occi/infrastructure#" class="kind" Category: medium scheme="http://fedcloud.egi.eu/occi/compute/flavour/1.0#" class="mixin" Category: uuid_training_jupyter_notebook_centos_6_fedcloud_warg_162 scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#" class="mixin" Category: compute scheme="http://opennebula.org/occi/infrastructure#" class="mixin" Category: user_data scheme="http://schemas.openstack.org/compute/instance#" class="mixin" Category: public_key scheme="http://schemas.openstack.org/instance/credentials#" class="mixin" X-OCCI-Attribute: occi.compute.cores=2 X-OCCI-Attribute: occi.compute.memory=2.0 X-OCCI-Attribute: occi.compute.state="active" X-OCCI-Attribute: occi.core.id=76623 X-OCCI-Attribute: occi.core.summary="Instantiated with rOCCI-server on Mon, 03 Oct 2016 17:02:26 +0200." X-OCCI-Attribute: occi.core.title="Jupyter" X-OCCI-Attribute: org.opennebula.compute.cpu=2.0 X-OCCI-Attribute: org.opennebula.compute.id=76623 X-OCCI-Attribute: org.openstack.compute.user_data="IyEvYmluL3NoCmVjaG8gIlRoaXMgaXMgYSBzaW1wbGUgQVNDSUkgZmlsZSEiID4gL3RtcC9sb2cudHh0CmVjaG8gIlRoZSB0aW1lIGlzIG5vdyAkKGRhdGUgLVIpISIgPj4gL3RtcC9sb2cudHh0Cg==" X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8rpuqrpp53owo/zbkgIM/oRyavJ+T7o1bxuI6XRugIQatBsr/njULHqEf+j59d4aaxFbSNL56waCpRngiowkY+rHzrwIDz62eP+CwqPxfkJcIgdikXaSVCW9iD5wF/PexU8fPzu2WYfamRNZyqnJSya5pVx0RxmxV++fqDLQT6HdZp0faITB0C4fKf1HLm1qfo0d0M2yKwShplzywpciQ7p05KbufxrK4udYQmOFBAaJwhUbEYkydO2bn55gf9kfBCgYdXxI9UWCslZ6vd54fagpmvA46+UGXnNuwCUYylDYw6FY0qEmchHH6HOzTrFmt+Rq+yJtK9j+ThU3w84tp ubuntu@egi-fedcloud-clients" Link: </network/24> rel="http://schemas.ogf.org/occi/infrastructure#network" self="/link/networkinterface/compute_76623_nic_0" category="http://schemas.ogf.org/occi/infrastructure#networkinterface http://opennebula.org/occi/infrastructure#networkinterface http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface" occi.core.id="compute_76623_nic_0" occi.core.source="/compute/76623" occi.core.target="/network/24" occi.core.title="public" occi.networkinterface.address="147.228.242.12" occi.networkinterface.interface="eth0" occi.networkinterface.mac="02:00:93:e4:f2:0c" occi.networkinterface.state="active" org.opennebula.networkinterface.bridge="onebr0" Link: </storage/3017> rel="http://schemas.ogf.org/occi/infrastructure#storage" self="/link/storagelink/compute_76623_disk_0" category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink" occi.core.id="compute_76623_disk_0" occi.core.source="/compute/76623" occi.core.target="/storage/3017" occi.core.title="20f1eac8-68b4-5938-b720-ab07b2edc485" occi.storagelink.deviceid="/dev/vda" occi.storagelink.state="active" Link: </compute/76623?action=restart> rel="http://schemas.ogf.org/occi/infrastructure/compute/action#restart" Link: </compute/76623?action=stop> rel="http://schemas.ogf.org/occi/infrastructure/compute/action#stop" Link: </compute/76623?action=suspend> rel="http://schemas.ogf.org/occi/infrastructure/compute/action#suspend"