Difference between revisions of "APEL/MessageFormat"
Line 13: | Line 13: | ||
The header only appears once at the top of each message. It defines the type of record and the schema version. | The header only appears once at the top of each message. It defines the type of record and the schema version. | ||
{| cellspacing="1" cellpadding="1" border="1" align="left" width=" | {| cellspacing="1" cellpadding="1" border="1" align="left" width="600" | ||
|- | |- | ||
! scope="col" | Key | ! scope="col" | Key | ||
! scope="col" | Value | ! scope="col" | Value | ||
! scope="col" | Description | ! scope="col" | Description | ||
! scope="col" | Mandatory | ! scope="col" | Mandatory | ||
|- | |- | ||
| Site | | Site | ||
| String | | String | ||
| GOCDB sitename | | GOCDB sitename | ||
| Yes | | Yes | ||
|- | |- | ||
| SubmitHost | | SubmitHost | ||
| String | | String | ||
| Head node where the job was submitted | | Head node where the job was submitted | ||
| Yes | | Yes | ||
|- | |- | ||
| LocalJobID | | LocalJobID | ||
| String | | String | ||
| Batch System Job ID | | Batch System Job ID | ||
| Yes | | Yes | ||
|- | |- | ||
| LocalUserID | | LocalUserID | ||
| String | | String | ||
| Local username | | Local username | ||
| | | | ||
|- | |- | ||
| GlobalUserName | | GlobalUserName | ||
| String | | String | ||
| User's X509 DN | | User's X509 DN | ||
| | | | ||
|- | |- | ||
| UserFQAN | | UserFQAN | ||
| String | | String | ||
| User's VOMS attributes | | User's VOMS attributes | ||
| | | | ||
|- | |- | ||
| WallDuration | | WallDuration | ||
| int | | int | ||
| Wallclock time for the job (seconds) | | Wallclock time for the job (seconds) | ||
| Yes | | Yes | ||
|- | |- | ||
| CpuDuration | | CpuDuration | ||
| int | | int | ||
| CPU time for the job (seconds) | | CPU time for the job (seconds) | ||
| Yes | | Yes | ||
|- | |- | ||
| Processors | | Processors | ||
| int | | int | ||
| Number of processors | | Number of processors | ||
| | | | ||
<br> | |||
|- | |- | ||
| NodeCount | | NodeCount | ||
| int | | int | ||
| Number of nodes | | Number of nodes | ||
| | | | ||
|- | |- | ||
| StartTime | | StartTime | ||
| int | | int | ||
| Start time of the job (epoch) | | Start time of the job (epoch) | ||
| Yes | | Yes | ||
|- | |- | ||
| EndTime | | EndTime | ||
| int | | int | ||
| Stop time of the job (epoch) | | Stop time of the job (epoch) | ||
| | | | ||
|- | |- | ||
| MemoryReal | | MemoryReal | ||
| int | | int | ||
| Memory consumed by job (kbytes) | | Memory consumed by job (kbytes) | ||
| | | | ||
|- | |- | ||
| MemoryVirtual | | MemoryVirtual | ||
| int | | int | ||
| Virtual memory consumed by job (kbytes) | | Virtual memory consumed by job (kbytes) | ||
| | | | ||
|- | |- | ||
| ScalingFactorUnit | | ScalingFactorUnit | ||
| String | | String | ||
| HepSpec | SpecInt | custom | | HepSpec | SpecInt | custom | ||
| Yes | | Yes | ||
|- | |- | ||
| ScalingFactor | | ScalingFactor | ||
| double | | double | ||
| Value of either HepSpec, SpecInt or custom | | Value of either HepSpec, SpecInt or custom | ||
| Yes | | Yes | ||
|} | |} | ||
'''End of record:''' %% | '''End of record:''' %% | ||
Line 110: | Line 113: | ||
ScalingFactor = 1 | ScalingFactor = 1 | ||
If !GlobalUserName or !UserFQAN is not published, the value for these fields on the server will be set to 'None'. | If !GlobalUserName or !UserFQAN is not published, the value for these fields on the server will be set to 'None'. | ||
=== Example Message === | === Example Message === |
Revision as of 13:05, 3 May 2011
APEL Message Format
This describes a new message format for getting data between the APEL clients and the server.
Job Records
A message can contain multiple records. Different records must be separated by the end of record marker (%%).
Description
Header APEL-individual-job-message: v0.1
The header only appears once at the top of each message. It defines the type of record and the schema version.
Key | Value | Description | Mandatory |
---|---|---|---|
Site | String | GOCDB sitename | Yes |
SubmitHost | String | Head node where the job was submitted | Yes |
LocalJobID | String | Batch System Job ID | Yes |
LocalUserID | String | Local username | |
GlobalUserName | String | User's X509 DN | |
UserFQAN | String | User's VOMS attributes | |
WallDuration | int | Wallclock time for the job (seconds) | Yes |
CpuDuration | int | CPU time for the job (seconds) | Yes |
Processors | int | Number of processors |
|
NodeCount | int | Number of nodes | |
StartTime | int | Start time of the job (epoch) | Yes |
EndTime | int | Stop time of the job (epoch) | |
MemoryReal | int | Memory consumed by job (kbytes) | |
MemoryVirtual | int | Virtual memory consumed by job (kbytes) | |
ScalingFactorUnit | String | HepSpec | SpecInt | custom | Yes |
ScalingFactor | double | Value of either HepSpec, SpecInt or custom | Yes |
End of record: %%
Notes: If !ScalingFactorUnit/Value is not available it should be set to:
ScalingFactorUnit = 'custom' ScalingFactor = 1
If !GlobalUserName or !UserFQAN is not published, the value for these fields on the server will be set to 'None'.
Example Message
{{{ APEL-individual-job-message: v0.1 Site: RAL-LCG2 SubmitHost: ce01.ncg.ingrid.pt:2119/jobmanager-lcgsge-atlasgrid LocalJobID: 31564872 LocalUserID: atlasprd019 GlobalUserName: /C=whatever/D=someDN UserFQAN: /voname/Role=NULL/Capability=NULL WallDuration: 234256 CpuDuration: 2345 Processors: 2 NodeCount: 2 StartTime: 1234567890 EndTime: 1234567899 MemoryReal: 1000 MemoryVirtual: 2000 ScalingFactorUnit: SpecInt2000 ScalingFactor: 1000 %% ...another job record... %% ... %% }}}
Summary Job Records
Description
Header: APEL-summary-job-message: v0.1
The header only appears once at the top of each message. It defines the type of record and the schema version. || Key || Value || Description || Mandatory || || Site || String || GOCDB/official name of the site where the job run || Yes || || Month || int || Month of summary || Yes || || Year || int || Year of summary || Yes || || !GlobalUserName || String || User's X.509 DN || || || VO || String || User's VO || || || Group || String || User's VOMS Group || || || Role || String ||User's VOMS Role || || || !WallDuration || int || clock time for the job || Yes || || !CpuDuration || int || CPU time for the job || || || !NormalisedCpuDuration || int || Normalised CPU time for the job || Yes || || !NormalisedWallDuration || int || Normalised Wall clock time || Yes || || !NumberOfJobs || int || Total number of jobs || Yes ||
End of record: %%
Notes:BR
If !GlobalUserName, VO, Role or Group are not published, the value for these fields on the server will be set to 'None'.BR
A single job record must only be included in one summary record to avoid duplication of data.BR
Example Message
{{{ APEL-summary-job-message: v0.1 Site: RAL-LCG2 Month: 3 Year: 2010 GlobalUserName: /C=whatever/D=someDN VO: atlas Group: /atlas Role: Role=production WallDuration: 234256 CpuDuration: 2345 NormalisedCpuDuration: 2500 NormalisedWallDuration: 244435 NumberOfJobs: 100 %% ...another summary job record... %% ... %% }}}
Summary Sync Records
The summary Sync records are used for the creation of the apel-sync Nagios test. It is a mechanism for the central APEL server to know the number of records that each site is storing locally.
Description
Header: APEL-sync-message: v0.1
The header only appears once at the top of each message. It defines the type of record and the schema version. || Key || Value || Description || Mandatory || || Site || String || GOCDB/official name of the site where the job run || Yes || || NJobs || int || Total number of jobs for that month || Yes || || NDays || int || Number of days between earliest and latest job in month || Yes || ||Month || int || Month || Yes || ||Year || int ||Year || Yes ||
End of record: %%
Notes:BR
Each record indicates the number of jobs run on the site per month. This data is used to create the Nagios apel-sync test.
Example Message
{{{ APEL-sync-message: v0.1 Site: RAL-LCG2 NJobs: 3479 NDays: 29 Month: 1 Year: 2010 %% ...another sync record... %% ... %% }}}