Difference between revisions of "APEL/Storage"

From EGIWiki
Jump to: navigation, search
(Configure GOCDB)
(Configure SSM)
Line 35: Line 35:
<code>network: PROD</code>
<code>network: PROD</code>
(Not this: <strike> <code>network: TEST-NWOB</code></strike>)
Also set:  
Also set:  

Revision as of 23:08, 19 September 2017

Storage space usage accounting in APEL is based on the StAR (Storage Accounting Record) developed by EMI in conjuction with the OGF Usage Record Work Group (UR-WG). The format is documented here.

EMI delivered StAR solutions for dCache and DPM in EMI-3. In both cases the storage service queries its database at a site and extracts data to populate StAR usage records. The site then uses SSM as a transport method to send the StAR records across the EGI Messaging Service to APEL's central accounting repository.

Deployment Instructions

Get the accounting script

Follow the link for your product:


DPM - you need to run 1.8.7 or higher. For earlier releases it can be found here http://svnweb.cern.ch/world/wsvn/lcgdm/lcg-dm/trunk/scripts/StAR-accounting/star-accounting.py

On the command line which runs the script, add: --site="your-GOCDB-sitename"

Get the SSM software

The APEL SSM software can be installed from the yum repo or by downloading an RPM or tarball. See here for more information: SSM

Get authorised

To authorise your host to publish storage accounting see the section on configuring GOCDB.

Configure GOCDB

You need to add a service endpoint for that host to GOCDB with the service type "eu.egi.storage.accounting" and the correct host certificate DN.

Configure SSM

To configure apel-ssm for sending accounting records edit the file /etc/apel/sender.cfg, and modify some parameters:

  • In the [broker] section, you need:

network: PROD

Also set:

use_ssl: true

  • In the [certificates] section, you need to set the host certificate to be used by the sender:

certificate: /path/to/your/host/certificate
key: /path/to/your/key

  • In the [messaging] section, you need:

For production publishing -> destination: /queue/global.accounting.storage.central

For test publishing -> destination: /queue/global.accounting.test.storage.central

As described here: ​https://wiki.egi.eu/wiki/APEL/SSM2Configuration.

path: set to the directory to which the StAR records are written by your storage software.

dCache and DPM should send one StAR record per VO. Please set up a cron to run once per day.

Running the Accounting Software

Create a cron job to run your accounting script followed by the SSM sender. We recommend that you send storage accounting data once per day. There will be a delay of up to 12 hours before you see the data you have sent reflected in the Accounting Portal.

Storage Data at the EGI Accounting Portal

Accounting Portal Prototype view