HOWTO12 Globus Online cookbook for EGI VOs

About

This page provides guidance for Virtual Organisations (VO) of the European Grid Infrastructure (EGI) on how to use the Globus Online service that is available at http://www.globusonline.eu. The services provides robust and easy to use file transfer capabilities for EGI users. Globus Online manages file transfers for you, monitoring performance, retrying failures, auto-tuning and recovering from faults automatically where possible, and reporting status. The Cookbook demonstrates how SRM storages of EGI could be used as endpoints for file transfers in Globus Online.

The Cookbook consists of two parts:

• The first part provides step-by-step instructions for VO Managers on how to register SRM storage services in Globus Online in such a way, that these appear as endpoints that VO members can find and use for file tranfers. Although these steps could be performed by any member of a VO, we assume that for most VOs the VO Manager is the most suitable person to complete the steps, because the VO Managers have sufficient knowledge on storage sites of the VO and about using the BDII information system to obtain detailed information about these storages.

• The second part provides step-by-step instructions for VO members on how to use storage endpoints in Globus Online. These steps can be carried out by any VO member.

The "biomed" VO is used as an example in the Cookbook to demonstrate Globus Online usage.

Please email any feedback about this Cookbook to the EGI.eu User Community Support Team: ucst@egi.eu.

ATTENTION: in the examples we use Glue v1.3 and Glue 2.0. Currently there several discussions ongoing about what storage managers must publish to information system and make easier way of querying the information. REF: https://ggus.eu/tech/ticket_show.php?ticket=94264

VO Managers

1. Get all storage managers (Glue 2.0):

-bash-3.2$ ldapsearch -x -h lcg-bdii.cern.ch -p 2170 -b o=glue '(&(objectclass=GLUE2StorageService))' |perl -p00e 's/\r?\n //g'|grep ^dn:|cut -d" " -f2 > GLUE2StorageServiceDN

2. Filter out only storage managers for your VO (Glue 2.0):

-bash-3.2$ for i in `cat GLUE2StorageServiceDN`;do ldapsearch -LLL -x -H ldap://lcg-bdii.cern.ch:2170 -x -b "$i" "(&(objectClass=GLUE2AccessPolicy)(GLUE2PolicyRule=*:biomed))" GLUE2PolicyRule| grep -q GLUE2PolicyRule && echo $i;done > GLUE2AccessPolicyDN

3. Get all GridFTP endpoints for your VO (Glue 2.0):

-bash-3.2$ for i in `cat GLUE2AccessPolicyDN`; do ldapsearch -LLL -x -H ldap://lcg-bdii.cern.ch:2170 -x -b "$i" "(&(objectClass=GLUE2Endpoint)(GLUE2EndpointInterfaceName=gsiftp))" GLUE2EndpointURL | grep ^GLUE |tail -n 1;done
GLUE2EndpointURL: gsiftp://gaeds023.ciemat.es:2811
GLUE2EndpointURL: gsiftp://se001.ipp.acad.bg:2811
GLUE2EndpointURL: gsiftp://juliet.zih.tu-dresden.de:2811
GLUE2EndpointURL: gsiftp://sedsk58.grid.hep.ph.ic.ac.uk:2811

Our tests with Globus Online were successful with DPM and dCache type of SRM storages, however the same process may work with some Storm storages (Storm is the third SRM storage type in use in EGI). With this query you can find out the type of the SRM storage that support your VO (GlueSEUniqueID=<SE_hostname>):