Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI Opendata platform

From EGIWiki
Revision as of 16:07, 20 September 2016 by Scardaci (talk | contribs) (→‎OneProvider)
Jump to navigation Jump to search
Overview For users For resource providers Infrastructure status Site-specific configuration Architecture




The EGI Open Data platform

Setup your ONEDATA deployment into the EGI Federated Cloud

This is a setep-by-step guide to setup a ONEDATA deployment into the EGI Federated Cloud. For further information about ONEDATA, please refer to its online documentation.

OneZone

ssh -i [path-to-your-private-key] ubuntu@[ip-address-of-your-instance] 
  • Install docker-compose in the machine as root.
sudo -i

curl -L https://github.com/docker/compose/releases/download/1.8.0-rc1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

and verify its setup:

~# docker-compose -v
docker-compose version 1.8.0-rc1, build 9bf6bc6

don't forget to exit from the root shell.

  • In the home of the ubuntu user download the OneData software:
git clone https://github.com/onedata/getting-started
  • Go to the scenario 3.0 folder.
cd getting-started/scenarios/3_0_oneprovider_onezone_multihost/
  • Install OneZone using the run_onedata.sh script:
sudo ./run_onedata.sh --zone &

Wait for the message "Congratulations! onezone has been successfully started."

Now your OneZone instance is ready. You can access it with your browser:

https://[ip-address-of-your-instance]

Please, login with

login:admin
password:password

OneProvider

  • Create your storage in the cloud and store its ID in an environment variable.
occi --endpoint $ENDPOINT  --auth x509 --voms --user-cred $X509_USER_PROXY --action create --resource storage
--attribute occi.storage.size="num([number-og-GB])" --attribute occi.core.title="[storage_name]"
[ID]

export STORAGE_ID=[ID]
  • Link the block storage to your instance.
occi --endpoint $ENDPOINT  --auth x509 --voms --user-cred $X509_USER_PROXY --action link --resource $COMPUTE_ID --link $STORAGE_ID
  • Run an OCCI describe command to your instance to get the block storage path. You can find it in the occi.storagelink.deviceid attribute. See the example below.


occi --endpoint $ENDPOINT  --auth x509 --voms --user-cred $X509_USER_PROXY --action describe --resource $COMPUTE_ID

[[ http://schemas.ogf.org/occi/infrastructure#compute ]]
>> location: /occi/compute/136618fb-4687-4a15-b144-5381d9aa0742
occi.core.id = 136618fb-4687-4a15-b144-5381d9aa0742
occi.core.title = onedata-provider100
occi.compute.cores = 2
occi.compute.hostname = onedata-provider100
occi.compute.memory = 4096
occi.compute.state = active

Links:

    [[ http://schemas.ogf.org/occi/core#link ]]
    >> location: /occi/storagelink/136618fb-4687-4a15-b144-5381d9aa0742_fd8e7b72-0c1e-4f9c-a1a0-d4043c56b752
    occi.storagelink.deviceid = /dev/vdb
    occi.core.source = http://cloud.recas.ba.infn.it:8787/occi/compute/136618fb-4687-4a15-b144-5381d9aa0742
    occi.core.target = http://cloud.recas.ba.infn.it:8787/occi/storage/fd8e7b72-0c1e-4f9c-a1a0-d4043c56b752
    occi.core.id = 136618fb-4687-4a15-b144-5381d9aa0742_fd8e7b72-0c1e-4f9c-a1a0-d4043c56b752

    [[ http://schemas.ogf.org/occi/core#link ]]
    >> location: /occi/networklink/136618fb-4687-4a15-b144-5381d9aa0742_fe82ef7b-4bb7-4c1e-b4ec-ec5c1b0c7333_90.147.102.160
    occi.networkinterface.mac = fa:16:3e:bf:26:94
    occi.networkinterface.interface = eth0
    occi.networkinterface.state = active
    occi.networkinterface.allocation = dynamic
    occi.networkinterface.address = 90.147.102.160
    occi.core.source = http://cloud.recas.ba.infn.it:8787/occi/compute/136618fb-4687-4a15-b144-5381d9aa0742
    occi.core.target = http://cloud.recas.ba.infn.it:8787/occi/network/fe82ef7b-4bb7-4c1e-b4ec-ec5c1b0c7333
    occi.core.id = 136618fb-4687-4a15-b144-5381d9aa0742_fe82ef7b-4bb7-4c1e-b4ec-ec5c1b0c7333_90.147.102.160

Mixins:

    [[ http://schemas.openstack.org/template/os#b7765c28-6bc6-438a-8b7c-b6873103c5f5 ]]
    title:        Image for Docker Ubuntu 14.04 [Ubuntu/14.04/VirtualBox]_EGI_fedcloud
    term:         b7765c28-6bc6-438a-8b7c-b6873103c5f5
    location:     /occi/os_tpl/b7765c28-6bc6-438a-8b7c-b6873103c5f5

    [[ http://schemas.openstack.org/template/resource#8 ]]
    title:        Flavor: medium
    term:         8
    location:     /occi/resource_tpl/8

Actions:

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#start ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#stop ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#restart ]]

    [[ http://schemas.ogf.org/occi/infrastructure/compute/action#suspend ]]

  • Login into the new instance with your private key.
ssh -i [path-to-your-private-key] ubuntu@[ip-address-of-your-instance] 
  • Format and mount the block storage attached to the instance
sudo mkfs.ext3 [block-storage-path-in-your-instance]
sudo mount /dev/vdb [choose-a-mount-point]

In the following an example with block storage identified by /dev/vdb and mount point /mnt/.

sudo mkfs.ext3 /dev/vdb
sudo mount /dev/vdb /mnt/
  • Install docker-compose in the machine as root.
sudo -i

curl -L https://github.com/docker/compose/releases/download/1.8.0-rc1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

and verify its setup:

~# docker-compose -v
docker-compose version 1.8.0-rc1, build 9bf6bc6

don't forget to exit from the root shell.

  • In the home of the ubuntu user download the OneData software:
git clone https://github.com/onedata/getting-started
  • go to the scenario 3.0 folder.
cd getting-started/scenarios/3_0_oneprovider_onezone_multihost/


  • Install OneZone using the run_onedata.sh script:

Access your data from an instance running into the EGI FedCloud