Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "MAN09 Accounting data publishing"

From EGIWiki
Jump to navigation Jump to search
Line 41: Line 41:


We recommend sending data daily for all sites, whether sending summaries or individual records. I think this is in the interests of people who are using the portal so that what they see is accurate. The summaries are for a complete month or the current month so far.  
We recommend sending data daily for all sites, whether sending summaries or individual records. I think this is in the interests of people who are using the portal so that what they see is accurate. The summaries are for a complete month or the current month so far.  
=== Testing broker  ===
The EGI run a network of message brokers which aim to be more resilient than a single message broker could be.
The production message broker hosts are:
* mq.cro-ngi.hr:6163
* mq.afroditi.hellasgrid.gr:6163
Site should not use SSL for testing. ''use_ssl:false''
The queue for testing is:
*/queue/global.accounting.test.cpu.central
**(case sensitive)


=== Authorization and authentication  ===
=== Authorization and authentication  ===

Revision as of 15:56, 17 September 2014

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators



Title Accounting data publishing
Document link https://wiki.egi.eu/wiki/MAN09
Last modified 19 August 2014
Policy Group Acronym OMB
Policy Group Name Operations Management Board
Contact Group operations-support@mailman.egi.eu
Document Status APPROVED
Approved Date 30.01.2014
Procedure Statement This manual provides information on how to publish accounting information from different middlewares.
Owner Owner of procedure


Introduction

In this manual we will show you how to publish accounting information from different middlewares.

General information

Publishing with the APEL Client

To start sending accounting records:

  1. register 'gLite-APEL' endpoint in GOCDB with host DN information
    • Changes in GOCDB can take up to 4 hours to make it to the message brokers.
  2. install the APEL client on your publisher host and edit client.cfg and sender.cfg
  3. install the APEL parser relevant to your batch system on each CE and edit parser.cfg
  4. configure the client to publish data from the start of the current month. 
  5. Run the parser(s) and client. 
  6. If there are no errors messages and the client says it has sent messages then wait a day and you should see jobs appearing here http://goc-accounting.grid-support.ac.uk/apel/jobs2.html or summaries here: http://goc-accounting.grid-support.ac.uk/apel/summaries2.html
  7. If it doesn't open a GGUS ticket for the APEL Team.

Publishing summarized records or single ones

Sites can send either. Larger sites are preferred to send summaries.

The frequency for sending aggregated/summary records to Apel database?

We recommend sending data daily for all sites, whether sending summaries or individual records. I think this is in the interests of people who are using the portal so that what they see is accurate. The summaries are for a complete month or the current month so far.

Authorization and authentication

Authorization and authentication is made by the host certificate (which is signed by a trusted CA from the ca_policy_core package). Certificate should be registered in GOC DB. The certificate must not have the x509 extension: "Netscape Cert Type: SSL Server" because the message brokers will reject it.


ARC

Instructions can be found here


Globus

The accounting system for the Globus Toolkit is called GridSAFE. There is an enhanced package provided by EGCF (formerly IGE). It is located in the EGCF repository for download/installation. There is a GridSAFE Service Reference Card as well.

The installation package contains information about how to parse your log-files to obtain the accounting information. Furthermore, there is a tool to create summary records in the APEL format which can be called using the ige-apel-client.sh script.

In order to publish your data to EGI, you need to install the APEL/SSM tool as well.

Multi-middleware site

No additional requirements.

QCG

For QCG Accounting installation and configuration please check The QCG-Accounting agent

Multi-middleware site

No additional requirements except: Services of each middlewere should be on separate machines.

Unicore

Publishing UNICORE jobs accounting data to EGI accounting system


Revision History

Version Authors Date Comments
M. Krakowian 19 August 2014 Change contact group -> Operations support