PROC22

From EGIWiki
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators


Title Support for CVMFS replication across the EGI Infrastructure
Document link https://wiki.egi.eu/wiki/PROC22
Last modified
Policy Group Acronym OMB
Policy Group Name Operations Management Board
Contact Group operations at egi.eu
Document Status DRAFT
Approved Date
Procedure Statement The procedure describes the process of creating a repository within the EGI CVMFS infrastructure for an EGI VO.
Owner Baptiste Grenier


Overview

The procedure describes the process of creating a repository within the EGI CVMFS infrastructure for an EGI VO.

Definitions

Please refer to the EGI Glossary for the definitions of the terms used in this procedure.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", “MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Entities involved in the procedure

  • VO manager
  • EGI Operations
  • EGI CVMFS Manager (EGIM)

Requirements

The prerequesites are:

  • VO is in production
  • VO is supported by more than one EGI production Resource Centres

Steps


Responsible Action Notes
1 VO manager (or VO SGM) create the request via GGUS ticket, specifying the name of the VO in EGI to be supported within the CVMFS infrastructure. The ticket should contain information about the initial size of the repository and, if possible, an expected growth rate.

For an initial size of 50GB+, the VO SGM might be asked for further details regarding the proposed structure of the repository.

Also the ticket must include the DNs (Distinguished Names) of VO users responsible for maintaining the repository in the future (all these VO users will be referred as VO SGM in this document). A VOMS Role could be provided alternatively.

Finally, email address of one or more SGMs for the VO must be provided, in order to contact them in case of infrastructure issues.

2 EGI Operations team verify that the request satisfies the prerequisites The prerequesites are:
  • VO is in production
  • VO Manager has been correctly involved in the ticket
  • VO is supported by more than one EGI production Resource Centres

The outcome of the validation must be reported in the ticket opened in Step 1.

If the request is valid the GGUS ticket must be assigned to the RAL-LCG2 site with cvmfs-support@gridpp.rl.ac.uk cc'ed.

3 EGI CVMFS Manager (EGIM) at RAL check if requests in the ticket opened in Step#1 are valid and proceeds to create the CVMFS repository on the EGI infrastructure (currently at RAL for the 'egi.eu' CVMFS domain). The repository will be defined in the form <repo_name>.egi.eu

, where <repo_name> should resemble the VO name where possible.

Also the EGIM provides access to VO SGM to the upload area (currently cvmfs-upload01.gridpp.rl.ac.uk ) and will communicate the required details for access and upload operations.

4 VO SGM set/check ability to upload/install software For uploading stuff (and maybe to unpack various tarballs) the VO SGM needs to access the GSI interface (gsissh / gsiscp tools) on (currently) cvmfs-upload01.gridpp.rl.ac.uk

server. For this the VO SGM needs beforehand to create a grid proxy on a User Interface machine (obviously containing the DN or the Role provided at Step#1). Then the gsiscp and gsissh connections will land them in a home directory; for some technicalities to access the CVMFS area the VO SGM has to 'cd ./cvmfs_repo' where one should be able to upload (using gsiscp) files and then unpack them there.


With gsissh / gsiscp one needs to use port 1975 ('gsissh -p 1975 cvmfs-upload01.gridpp.rl.ac.uk ')


And one important thing: all files and directories part of a CVMFS repository must be 'read'-able by everyone i.e. to be on the safe side of things every single time one updated the repository he/she would need to


  1. find ./ -type d -exec chmod go+rx {} \;


  1. find ./ -type f -exec chmod go+r {} \;


unless the files/directories are uploaded already with the appropriate permissions.


If the bundle contains any non-readable files, it will not publish!!!

5 EGIM check if the repository was correctly published and initiate the replication at the EGI Stratum-1 network level
6 Sites access data by any CVMFS client configured to support the egi.eu Once the repository has been replicated by at least one of the supporting Stratum-1 (usually RAL Stratum-1 is the first), the data can be accessed by any CVMFS client at a site configured to support the egi.eu

CVMFS space.

The only additional thing needed is the (re)-definition of the VO_SW_DIR variable i.e. VO_<vo_name>_SW_DIR=/cvmfs/<repo_name>.egi.eu , at least for grid sites. And this requirement could also be added to the VO ID card.

Otherwise, the /cvmfs/<repo_name>.egi.eu space should be readable by direct access.

Revision History

Version Authors Date Comments
1.0 Vincenzo Spinoso, Catalin Condurache December 18, 2015
Alessandro Paolini 2016-06-08 "EGI Operations Support" was decommissioned, changed all the references to "Operations"