Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "APEL/Server"

From EGIWiki
Jump to navigation Jump to search
Line 42: Line 42:


== Summarizer ==
== Summarizer ==
This processes the Job Records and Summary Records to create the CPU, User Summaries and SpecIntHistory tables required by the portal.
This processes the Job Records and Summary Records to create the CPU, User SuperSummaries and SpecIntHistory tables required by the portal.


== Record publisher ==
== Record publisher ==

Revision as of 12:18, 18 January 2012

This page is under construction and is not yet complete.

New APEL Overview

  • Sites which do not run the APEL client will send summary records via SSM which will be inserted in the Summaries table in the central APEL database.
  • The APEL clients will send Job records via SSM which will be inserted in the JobRecords table in the central APEL database.
  • Summary records will be created from the data in the JobRecords table. These JobRecords Summaries will be loaded into a SuperSummaries table (the SuperSummaries table will have the same schema as the Summaries table).
  • The records from external clients in the Summaries table will be inserted (replaced) into the SuperSummaries table.
  • The SpecInt data will also be extracted and stored in a SpecInt table.
  • The relevant Summary and SpecInt data will then be sent to the Accounting Portal using another instance of the SSM.

APEL Server Design

The new APEL server is written in Python, the database is MySQL. It has the following components:

  • SSM
  • Record loader
  • Database
  • Summarizer
  • Record publisher
  • Authentication
  • Monitoring

APEL Message Format

SSM

The SSM is the messaging system used by APEL to transmit messages. It is written in Python and uses the STOMP protocol.

External SSM Testing

APEL SSM Testing: If you are interested in testing the new STOMP interface or want to develop your own client.

Record loader

This has the task of taking the received messages and inserting the records in the database. At present there are three message types defined:

  • Job Record messages
  • Summary Record messages
  • Sync Record messages

The header in the message defines the type of records the message will contain and the version of the format. This will enable us to identify newer versions if they are defined.

This simple model allows the definition of alternative message types for different records which the record loader can then be configured to handle, for example, there will be a SpecIntHistory Record message type which is a requirement of the Accounting portal.

Summarizer

This processes the Job Records and Summary Records to create the CPU, User SuperSummaries and SpecIntHistory tables required by the portal.

Record publisher

This takes the information from a database and produces messages to be sent to the portal from the server using an instance of the SSM. (The Record publisher will also be used by the new version of the APEL client to create messages from the APEL client database to be sent to the APEL server.)

Authentication

This verifies that the senders of the messages via the SSM are known to APEL.