Difference between revisions of "EGI Opendata platform"
Jump to navigation
Jump to search
Line 199: | Line 199: | ||
* Install | * Install OneProvider using the run_onedata.sh script: | ||
<pre> | <pre> | ||
sudo ./run_onedata.sh --provider --provider-fqdn [OneZone-IP] --zone-fqdn [OneData-IP] & | sudo ./run_onedata.sh --provider --provider-fqdn [OneZone-IP] --zone-fqdn [OneData-IP] & | ||
</pre> | </pre> | ||
Wait for the message "Congratulations! oneprovider has been successfully started." | |||
See below an example of output. | |||
<pre> | <pre> | ||
IMPORTANT: After each start wait for a message: Congratulations! oneprovider has been successfully started. | |||
To ensure that the oneprovider is completely setup. | |||
Pulling node1.oneprovider.localhost (onedata/oneprovider:3.0.0-rc5)... | |||
3.0.0-rc5: Pulling from onedata/oneprovider | |||
a3ed95caeb02: Pull complete | |||
57c018ee1aa8: Pull complete | |||
ebd817e34e50: Pull complete | |||
c068f2d0906a: Pull complete | |||
b3bad60afaef: Pull complete | |||
47175703b6bc: Pull complete | |||
5cd40b9132e8: Pull complete | |||
25c596923cc2: Pull complete | |||
e665a356e4f5: Pull complete | |||
6585e20d8e6f: Pull complete | |||
229064f88987: Pull complete | |||
debcc3142260: Pull complete | |||
0056c9a8edc5: Pull complete | |||
Digest: sha256:88c0fd1fa61ebd4e548222beb14f9d29efacddcf0d8bea7f1f1185d4d453be64 | |||
Status: Downloaded newer image for onedata/oneprovider:3.0.0-rc5 | |||
Creating network "30oneprovideronezonemultihost_default" with the default driver | |||
Creating oneprovider-1 | |||
Attaching to oneprovider-1 | |||
oneprovider-1 | Starting op_panel: [ OK ] | |||
oneprovider-1 | | |||
oneprovider-1 | Configuring oneprovider: | |||
oneprovider-1 | * service_onepanel: set_cookie | |||
oneprovider-1 | * service_onepanel: purge_node | |||
oneprovider-1 | * service_onepanel: create_tables | |||
oneprovider-1 | * service_onepanel: add_default_users | |||
oneprovider-1 | * service_onepanel: add_nodes | |||
oneprovider-1 | * service: save | |||
oneprovider-1 | * service_couchbase: configure | |||
oneprovider-1 | * service_couchbase: start | |||
oneprovider-1 | * service_couchbase: wait_for_init | |||
oneprovider-1 | * service_couchbase: init_cluster | |||
oneprovider-1 | * service_couchbase: rebalance_cluster | |||
oneprovider-1 | * service_couchbase: status | |||
oneprovider-1 | * service: save | |||
oneprovider-1 | * service_cluster_manager: configure | |||
oneprovider-1 | * service_cluster_manager: start | |||
oneprovider-1 | * service_cluster_manager: status | |||
oneprovider-1 | * service: save | |||
oneprovider-1 | * service_op_worker: configure | |||
oneprovider-1 | * service_op_worker: setup_certs | |||
oneprovider-1 | * service_op_worker: start | |||
oneprovider-1 | * service_op_worker: wait_for_init | |||
oneprovider-1 | * service_op_worker: status | |||
oneprovider-1 | * service_op_worker: add_storages | |||
oneprovider-1 | * service_oneprovider: configure | |||
oneprovider-1 | * service_oneprovider: configure | |||
oneprovider-1 | * service_oneprovider: register | |||
oneprovider-1 | * service: save | |||
oneprovider-1 | * service_onepanel: add_users | |||
oneprovider-1 | | |||
oneprovider-1 | Container details: | |||
oneprovider-1 | * IP Address: 172.18.0.2 | |||
oneprovider-1 | * Ports: - | |||
oneprovider-1 | | |||
oneprovider-1 | Congratulations! oneprovider has been successfully started. | |||
</pre> | </pre> | ||
=== Access your data from an instance running into the EGI FedCloud === | === Access your data from an instance running into the EGI FedCloud === |
Revision as of 17:16, 20 September 2016
Overview | For users | For resource providers | Infrastructure status | Site-specific configuration | Architecture |
The EGI Open Data platform
Setup your ONEDATA deployment into the EGI Federated Cloud
This is a setep-by-step guide to setup a ONEDATA deployment into the EGI Federated Cloud. For further information about ONEDATA, please refer to its online documentation.
OneZone
- Create an instance using the EGI Docker (Ubuntu 14.04) image. Please, refer to the cli guide.
- Login into the new instance with your private key.
ssh -i [path-to-your-private-key] ubuntu@[ip-address-of-your-instance]
- Install docker-compose in the machine as root.
sudo -i curl -L https://github.com/docker/compose/releases/download/1.8.0-rc1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose
and verify its setup:
~# docker-compose -v docker-compose version 1.8.0-rc1, build 9bf6bc6
don't forget to exit from the root shell.
- In the home of the ubuntu user download the OneData software:
git clone https://github.com/onedata/getting-started
- Go to the scenario 3.0 folder.
cd getting-started/scenarios/3_0_oneprovider_onezone_multihost/
- Install OneZone using the run_onedata.sh script:
sudo ./run_onedata.sh --zone &
Wait for the message "Congratulations! onezone has been successfully started."
Now your OneZone instance is ready. You can access it with your browser:
https://[ip-address-of-your-instance]
Please, login with
login:admin password:password
OneProvider
- Create an instance using the EGI Docker (Ubuntu 14.04) image. Please, refer to the cli guide.
- Create your storage in the cloud and store its ID in an environment variable.
occi --endpoint $ENDPOINT --auth x509 --voms --user-cred $X509_USER_PROXY --action create --resource storage --attribute occi.storage.size="num([number-og-GB])" --attribute occi.core.title="[storage_name]" [ID] export STORAGE_ID=[ID]
- Link the block storage to your instance.
occi --endpoint $ENDPOINT --auth x509 --voms --user-cred $X509_USER_PROXY --action link --resource $COMPUTE_ID --link $STORAGE_ID
- Run an OCCI describe command to your instance to get the block storage path. You can find it in the occi.storagelink.deviceid attribute. See the example below.
occi --endpoint $ENDPOINT --auth x509 --voms --user-cred $X509_USER_PROXY --action describe --resource $COMPUTE_ID [[ http://schemas.ogf.org/occi/infrastructure#compute ]] >> location: /occi/compute/136618fb-4687-4a15-b144-5381d9aa0742 occi.core.id = 136618fb-4687-4a15-b144-5381d9aa0742 occi.core.title = onedata-provider100 occi.compute.cores = 2 occi.compute.hostname = onedata-provider100 occi.compute.memory = 4096 occi.compute.state = active Links: [[ http://schemas.ogf.org/occi/core#link ]] >> location: /occi/storagelink/136618fb-4687-4a15-b144-5381d9aa0742_fd8e7b72-0c1e-4f9c-a1a0-d4043c56b752 occi.storagelink.deviceid = /dev/vdb occi.core.source = http://cloud.recas.ba.infn.it:8787/occi/compute/136618fb-4687-4a15-b144-5381d9aa0742 occi.core.target = http://cloud.recas.ba.infn.it:8787/occi/storage/fd8e7b72-0c1e-4f9c-a1a0-d4043c56b752 occi.core.id = 136618fb-4687-4a15-b144-5381d9aa0742_fd8e7b72-0c1e-4f9c-a1a0-d4043c56b752 [[ http://schemas.ogf.org/occi/core#link ]] >> location: /occi/networklink/136618fb-4687-4a15-b144-5381d9aa0742_fe82ef7b-4bb7-4c1e-b4ec-ec5c1b0c7333_90.147.102.160 occi.networkinterface.mac = fa:16:3e:bf:26:94 occi.networkinterface.interface = eth0 occi.networkinterface.state = active occi.networkinterface.allocation = dynamic occi.networkinterface.address = 90.147.102.160 occi.core.source = http://cloud.recas.ba.infn.it:8787/occi/compute/136618fb-4687-4a15-b144-5381d9aa0742 occi.core.target = http://cloud.recas.ba.infn.it:8787/occi/network/fe82ef7b-4bb7-4c1e-b4ec-ec5c1b0c7333 occi.core.id = 136618fb-4687-4a15-b144-5381d9aa0742_fe82ef7b-4bb7-4c1e-b4ec-ec5c1b0c7333_90.147.102.160 Mixins: [[ http://schemas.openstack.org/template/os#b7765c28-6bc6-438a-8b7c-b6873103c5f5 ]] title: Image for Docker Ubuntu 14.04 [Ubuntu/14.04/VirtualBox]_EGI_fedcloud term: b7765c28-6bc6-438a-8b7c-b6873103c5f5 location: /occi/os_tpl/b7765c28-6bc6-438a-8b7c-b6873103c5f5 [[ http://schemas.openstack.org/template/resource#8 ]] title: Flavor: medium term: 8 location: /occi/resource_tpl/8 Actions: [[ http://schemas.ogf.org/occi/infrastructure/compute/action#start ]] [[ http://schemas.ogf.org/occi/infrastructure/compute/action#stop ]] [[ http://schemas.ogf.org/occi/infrastructure/compute/action#restart ]] [[ http://schemas.ogf.org/occi/infrastructure/compute/action#suspend ]]
- Login into the new instance with your private key.
ssh -i [path-to-your-private-key] ubuntu@[ip-address-of-your-instance]
- Format and mount the block storage attached to the instance
sudo mkfs.ext3 [block-storage-path-in-your-instance] sudo mount /dev/vdb [mount-point]
In the following an example with block storage identified by /dev/vdb and mount point /mnt/.
sudo mkfs.ext3 /dev/vdb sudo mount /dev/vdb /mnt/
- Install docker-compose in the machine as root.
sudo -i curl -L https://github.com/docker/compose/releases/download/1.8.0-rc1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose
and verify its setup:
~# docker-compose -v docker-compose version 1.8.0-rc1, build 9bf6bc6
don't forget to exit from the root shell.
- In the home of the ubuntu user download the OneData software:
git clone https://github.com/onedata/getting-started
- go to the scenario 3.0 folder.
cd getting-started/scenarios/3_0_oneprovider_onezone_multihost/
- edit file docker-compose-oneprovider.yml and replace ${ONEPROVIDER_DATA_DIR} with the folder where you mounted the block storage.
# data persistence - "[mount-point]:/volumes/storage"
- Install OneProvider using the run_onedata.sh script:
sudo ./run_onedata.sh --provider --provider-fqdn [OneZone-IP] --zone-fqdn [OneData-IP] &
Wait for the message "Congratulations! oneprovider has been successfully started."
See below an example of output.
IMPORTANT: After each start wait for a message: Congratulations! oneprovider has been successfully started. To ensure that the oneprovider is completely setup. Pulling node1.oneprovider.localhost (onedata/oneprovider:3.0.0-rc5)... 3.0.0-rc5: Pulling from onedata/oneprovider a3ed95caeb02: Pull complete 57c018ee1aa8: Pull complete ebd817e34e50: Pull complete c068f2d0906a: Pull complete b3bad60afaef: Pull complete 47175703b6bc: Pull complete 5cd40b9132e8: Pull complete 25c596923cc2: Pull complete e665a356e4f5: Pull complete 6585e20d8e6f: Pull complete 229064f88987: Pull complete debcc3142260: Pull complete 0056c9a8edc5: Pull complete Digest: sha256:88c0fd1fa61ebd4e548222beb14f9d29efacddcf0d8bea7f1f1185d4d453be64 Status: Downloaded newer image for onedata/oneprovider:3.0.0-rc5 Creating network "30oneprovideronezonemultihost_default" with the default driver Creating oneprovider-1 Attaching to oneprovider-1 oneprovider-1 | Starting op_panel: [ OK ] oneprovider-1 | oneprovider-1 | Configuring oneprovider: oneprovider-1 | * service_onepanel: set_cookie oneprovider-1 | * service_onepanel: purge_node oneprovider-1 | * service_onepanel: create_tables oneprovider-1 | * service_onepanel: add_default_users oneprovider-1 | * service_onepanel: add_nodes oneprovider-1 | * service: save oneprovider-1 | * service_couchbase: configure oneprovider-1 | * service_couchbase: start oneprovider-1 | * service_couchbase: wait_for_init oneprovider-1 | * service_couchbase: init_cluster oneprovider-1 | * service_couchbase: rebalance_cluster oneprovider-1 | * service_couchbase: status oneprovider-1 | * service: save oneprovider-1 | * service_cluster_manager: configure oneprovider-1 | * service_cluster_manager: start oneprovider-1 | * service_cluster_manager: status oneprovider-1 | * service: save oneprovider-1 | * service_op_worker: configure oneprovider-1 | * service_op_worker: setup_certs oneprovider-1 | * service_op_worker: start oneprovider-1 | * service_op_worker: wait_for_init oneprovider-1 | * service_op_worker: status oneprovider-1 | * service_op_worker: add_storages oneprovider-1 | * service_oneprovider: configure oneprovider-1 | * service_oneprovider: configure oneprovider-1 | * service_oneprovider: register oneprovider-1 | * service: save oneprovider-1 | * service_onepanel: add_users oneprovider-1 | oneprovider-1 | Container details: oneprovider-1 | * IP Address: 172.18.0.2 oneprovider-1 | * Ports: - oneprovider-1 | oneprovider-1 | Congratulations! oneprovider has been successfully started.