Difference between revisions of "APEL/Storage"

From EGIWiki
Jump to: navigation, search
m (Storage Data at the EGI Accounting Portal)
m (Storage Data at the EGI Accounting Portal)
Line 56: Line 56:
== Storage Data at the EGI Accounting Portal ==
== Storage Data at the EGI Accounting Portal ==
[http://accounting-devel.egi.eu/storage.php? Accounting Portal Prototype view] 
[http://accounting-devel.egi.eu/storage.php? Accounting Portal Prototype view] 
[http://goc-accounting.grid-support.ac.uk/storagetest/storagesitesystems.html A list of sites publishing storage accounting, with last publish dates.] 

Revision as of 13:56, 6 December 2016

Storage accounting in APEL is based on the StAR (Storage Accounting Record) developed by EMI in conjuction with the OGF Usage Record Work Group (UR-WG). The format is documented here.

EMI delivered StAR solutions for dcache and DPM in EMI-3. In both cases the storage service queries its database at a site and extracts data to populate StAR usage records. The site then uses SSM as a transport method to send the StAR records across the EGI Messaging Service to APEL's central accounting repository. 

Deployment Instructions

Get the accounting script

Follow the link for your product.


DPM - you need to run 1.8.7 or higher. For earlier releases it can be found here http://svnweb.cern.ch/world/wsvn/lcgdm/lcg-dm/trunk/scripts/StAR-accounting/star-accounting.py

On the command line which runs the script, add --site="your-GOCDB-sitename"

Get the SSM software

The APEL SSM software can be installed from the yum repo or by downloading an rpm or tarball. See here for more information: SSM

Get authorised

While storage accounting publishing is still in test mode please send the host certificate DN to apel-admins@stfc.ac.uk and ask them to authorise you for storage publishing.

Configure SSM

To configure apel-ssm for sending acounting records we have to edit the file /etc/apel/sender.cfg, and modify some parameters:

  • In the [broker] section, you need:

network: PROD and not: network: TEST-NWOB as for initial testing. Also set:

use_ssl: false

  • In the [certificates] section, you need to set the host certificate to be used by the sender:

certificate: /path/to/your/host/certificate
key: /path/to/your/key

  • In the [messaging] section, you need:

destination: /queue/global.accounting.test.storage.central - as described here: ​https://wiki.egi.eu/wiki/APEL/SSM2Configuration.
path: set to the directory to which the StAR records are written by your storage software.

dcache and dpm should send one StAR record per VO. Please set up a cron to run once per day.

Configure GOCDB

When storage accounting moves to the production message broker network then you will need to add a servicetype to GOCDB for the host which runs SSM. The servicetype has not yet been defined but it will require the host certificate DN.

Running the Accounting Software

Create a cron job to run your accounting script followed by the SSM sender. We recommend that you send storage accounting data once per day. There will be a delay of up to 24 hours before you see the data you have sent in the Accounting Portal.

Storage Data at the EGI Accounting Portal

Accounting Portal Prototype view