https://wiki.egi.eu/w/api.php?action=feedcontributions&user=Wrogers&feedformat=atomEGIWiki - User contributions [en]2024-03-29T07:19:51ZUser contributionsMediaWiki 1.37.1https://wiki.egi.eu/w/index.php?title=SSM_Migration_Status&diff=54684SSM Migration Status2013-05-01T15:52:15Z<p>Wrogers: </p>
<hr />
<div>The table below shows the status of all the systems under migration to SSM and their status <br />
<br />
{| class="wikitable"<br />
|-<br />
! System <br />
! Contacts <br />
! Status<br />
|-<br />
| CERN <br />
| Jérôme Belleman &lt;Jerome.Belleman@CERN.CH&gt; <br />
| In production<br />
|-<br />
| OSG - Gratia <br />
| John Wiegand &lt;weigand@fnal.gov&gt; Tanya Levshina &lt;tlevshin@fnal.gov&gt; <br />
| In production<br />
|-<br />
| SGAS - NDGF <br />
| Erik Edelman &lt;edelmann@csc.fi&gt; <br />
| In production<br />
|-<br />
| SGAS - Switzerland <br />
| Placi Flury &lt;placi.flury@switch.ch&gt; <br />
| In production<br />
|-<br />
| DGAS <br />
| Andrea Guarise &lt;andrea.guarise@to.infn.it&gt; <br />
| In production<br />
|-<br />
| NIKHEF <br />
| David Groep &lt;davidg@nikhef.nl&gt; <br />
| In production<br />
|-<br />
| CC-IN2P3 <br />
| Cecile Evesque &lt;cecile.evesque@cc.in2p3.fr&gt; <br />
| In production<br />
|-<br />
| SGAS - Finland <br />
| Ulf Tigerstedt &lt;ulf.tigerstedt@CSC.FI&gt; <br />
| status unknown<br />
|-<br />
| JURA (ArcCE) <br />
| Gabor Szigeti &lt;szigeti@niif.hu&gt; <br />
| Successfully tested using SSM2.<br />
|-<br />
| Unicore <br />
| Krzysztof Benedyczak &lt;golbi@icm.edu.pl&gt; <br />
| Successfully tested with SSM2-compatible software.<br />
|-<br />
| QosCosGrid/MAPPER <br />
| Mariusz Mamonski &lt;mamonski@man.poznan.pl&gt; <br />
| Done an analysis. Tested successfully with SSM1; will update to SSM2.<br />
|-<br />
| EDGI <br />
| Robert.Lovas@sztaki.mta.hu, Kitti Varga &lt;vargaki@sztaki.hu&gt; <br />
| <br />
Plan for collecting information in their CE <br />
<br />
(T<span class="comment-text"><span><span style="white-space: pre-wrap;">he activity is re-scheduled due to the significant delay of starting the related project (IDGF-SP project kick-off meeting was held last month). However, we are planning to continue this work as soon as possible, and report its status at the upcoming community forum in Manchester.</span></span></span>) <br />
<br />
|-<br />
| IGE/GridSafe <br />
| Stephen Crouch &lt;stc@ecs.soton.ac.uk&gt; <br />
| delivered in September 2012. Testing performed by Tilo Eißler and successful. UPDATE 21-03-2013: GridSAFE has been successfully tested including the upload of usage records to EGI. I'm currently waiting for the next IGE GridSAFE package which will contain some extensions (VO support) before bringing the LRZ test system into production. But this will be very soon.<br />
|-<br />
| PRACE <br />
| Jules Wolfrat, Stephen Booth <br />
| Analysis but Gridsafe should be reusable from IGE.<br />
|}<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51928APEL/DatabaseTables2013-02-20T13:02:53Z<p>Wrogers: /* Sync Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || bigint || || || || || <br />
|-<br />
| CpuDuration || bigint || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || No || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(20) || || || || ||<br />
|-<br />
| MemoryReal || bigint || || || || || <br />
|-<br />
| MemoryVirtual || bigint || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || decimal || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| UpdateTime || TimeStamp || Auto || || ||<br />
|-<br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|- <br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory View ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Key<br />
|-<br />
| UpdateTime || TimeStamp || <br />
|- <br />
| Site || varchar(255) || MUL <br />
|-<br />
| HepSpec06 || int || MUL <br />
|-<br />
| NumberOfJobs || int || <br />
|-<br />
| Month || int || MUL <br />
|- <br />
| Year || int || MUL <br />
|-<br />
| EarliestEndDate || date || <br />
|-<br />
| LatestEndDate || date || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51927APEL/DatabaseTables2013-02-20T13:02:20Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || bigint || || || || || <br />
|-<br />
| CpuDuration || bigint || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || No || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(20) || || || || ||<br />
|-<br />
| MemoryReal || bigint || || || || || <br />
|-<br />
| MemoryVirtual || bigint || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || decimal || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| UpdateTime || TimeStamp || Auto || || ||<br />
|-<br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|- <br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory View ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Key<br />
|-<br />
| UpdateTime || TimeStamp || <br />
|- <br />
| Site || varchar(255) || MUL <br />
|-<br />
| HepSpec06 || int || MUL <br />
|-<br />
| NumberOfJobs || int || <br />
|-<br />
| Month || int || MUL <br />
|- <br />
| Year || int || MUL <br />
|-<br />
| EarliestEndDate || date || <br />
|-<br />
| LatestEndDate || date || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51926APEL/DatabaseTables2013-02-20T13:00:52Z<p>Wrogers: /* HepSpecHistory Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || bigint || || || || || <br />
|-<br />
| CpuDuration || bigint || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || No || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(20) || || || || ||<br />
|-<br />
| MemoryReal || bigint || || || || || <br />
|-<br />
| MemoryVirtual || bigint || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || decimal || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory View ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Key<br />
|-<br />
| UpdateTime || TimeStamp || <br />
|- <br />
| Site || varchar(255) || MUL <br />
|-<br />
| HepSpec06 || int || MUL <br />
|-<br />
| NumberOfJobs || int || <br />
|-<br />
| Month || int || MUL <br />
|- <br />
| Year || int || MUL <br />
|-<br />
| EarliestEndDate || date || <br />
|-<br />
| LatestEndDate || date || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51925APEL/DatabaseTables2013-02-20T13:00:14Z<p>Wrogers: /* HepSpecHistory Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || bigint || || || || || <br />
|-<br />
| CpuDuration || bigint || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || No || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(20) || || || || ||<br />
|-<br />
| MemoryReal || bigint || || || || || <br />
|-<br />
| MemoryVirtual || bigint || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || decimal || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Key<br />
|-<br />
| UpdateTime || TimeStamp || ||<br />
|- <br />
| Site || varchar(255) || MUL || <br />
|-<br />
| HepSpec06 || int || MUL ||<br />
|-<br />
| NumberOfJobs || int || ||<br />
|-<br />
| Month || int || MUL ||<br />
|- <br />
| Year || int || MUL ||<br />
|-<br />
| EarliestEndDate || date || ||<br />
|-<br />
| LatestEndDate || date || ||<br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51924APEL/DatabaseTables2013-02-20T12:15:28Z<p>Wrogers: /* JobRecords Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || bigint || || || || || <br />
|-<br />
| CpuDuration || bigint || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || No || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(20) || || || || ||<br />
|-<br />
| MemoryReal || bigint || || || || || <br />
|-<br />
| MemoryVirtual || bigint || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || decimal || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || PRI || || <br />
|-<br />
| HepSpec06 || int || No || PRI || ||<br />
|-<br />
| NumberOfJobs || int || No || || 'None' || <br />
|-<br />
| Month || int || No || PRI || || <br />
|- <br />
| Year || int || No || PRI || || <br />
|-<br />
| EarliestEndDate || date || No || || 'None' ||<br />
|-<br />
| LatestEndDate || date || No || || 'None' || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51923APEL/DatabaseTables2013-02-20T12:14:59Z<p>Wrogers: /* JobRecords Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || int || || || || || <br />
|-<br />
| CpuDuration || int || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || No || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(20) || || || || ||<br />
|-<br />
| MemoryReal || bigint || || || || || <br />
|-<br />
| MemoryVirtual || bigint || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || decimal || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || PRI || || <br />
|-<br />
| HepSpec06 || int || No || PRI || ||<br />
|-<br />
| NumberOfJobs || int || No || || 'None' || <br />
|-<br />
| Month || int || No || PRI || || <br />
|- <br />
| Year || int || No || PRI || || <br />
|-<br />
| EarliestEndDate || date || No || || 'None' ||<br />
|-<br />
| LatestEndDate || date || No || || 'None' || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51922APEL/DatabaseTables2013-02-20T12:13:14Z<p>Wrogers: /* JobRecords Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || int || || || || || <br />
|-<br />
| CpuDuration || int || || || || || <br />
|-<br />
| Processors || int || No || || 0 || || <br />
|-<br />
| NodeCount || int || No || || 0 || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(255) || || || || ||<br />
|-<br />
| MemoryReal || int || || || || || <br />
|-<br />
| MemoryVirtual || int || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || double || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || PRI || || <br />
|-<br />
| HepSpec06 || int || No || PRI || ||<br />
|-<br />
| NumberOfJobs || int || No || || 'None' || <br />
|-<br />
| Month || int || No || PRI || || <br />
|- <br />
| Year || int || No || PRI || || <br />
|-<br />
| EarliestEndDate || date || No || || 'None' ||<br />
|-<br />
| LatestEndDate || date || No || || 'None' || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51921APEL/DatabaseTables2013-02-20T12:12:55Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || int || || || || || <br />
|-<br />
| CpuDuration || int || || || || || <br />
|-<br />
| Processors || int || No || 0 || || || <br />
|-<br />
| NodeCount || int || No || 0 || || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(255) || || || || ||<br />
|-<br />
| MemoryReal || int || || || || || <br />
|-<br />
| MemoryVirtual || int || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || double || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || PRI || || <br />
|-<br />
| HepSpec06 || int || No || PRI || ||<br />
|-<br />
| NumberOfJobs || int || No || || 'None' || <br />
|-<br />
| Month || int || No || PRI || || <br />
|- <br />
| Year || int || No || PRI || || <br />
|-<br />
| EarliestEndDate || date || No || || 'None' ||<br />
|-<br />
| LatestEndDate || date || No || || 'None' || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51920APEL/DatabaseTables2013-02-20T12:11:45Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || Yes ||<br />
|-<br />
| Queue || varchar(255) || No || || || Yes ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || int || || || || || <br />
|-<br />
| CpuDuration || int || || || || || <br />
|-<br />
| Processors || int || || || || || <br />
|-<br />
| NodeCount || int || || || || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(255) || || || || ||<br />
|-<br />
| MemoryReal || int || || || || || <br />
|-<br />
| MemoryVirtual || int || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || double || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || PRI || || <br />
|-<br />
| HepSpec06 || int || No || PRI || ||<br />
|-<br />
| NumberOfJobs || int || No || || 'None' || <br />
|-<br />
| Month || int || No || PRI || || <br />
|- <br />
| Year || int || No || PRI || || <br />
|-<br />
| EarliestEndDate || date || No || || 'None' ||<br />
|-<br />
| LatestEndDate || date || No || || 'None' || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/DatabaseTables&diff=51919APEL/DatabaseTables2013-02-20T12:10:34Z<p>Wrogers: /* JobRecords Table */</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
* See also [[APEL/MessageFormat]] - this will have more explanation of what the fields mean.<br />
<br />
These are here for reference. Note that MUL beside multiple columns refers to a composite key.<br />
<br />
== JobRecords Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="600"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
! scope="col" | Indexed<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || || <br />
|- <br />
| Site || varchar(255) || No || || || Yes || Yes <br />
|-<br />
| SubmitHost || varchar(255) || No || MUL || || Yes ||<br />
|-<br />
| MachineName || varchar(255) || No || || || ||<br />
|-<br />
| Queue || varchar(255) || No || || || ||<br />
|-<br />
| LocalJobId || varchar(255) || No || MUL || || ||<br />
|-<br />
| LocalUserId || varchar(255) || || || || ||<br />
|-<br />
| GlobalUserName || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| FQAN || varchar(255) || || || || || <br />
|-<br />
| VO || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| VORole || varchar(255) || No || || 'None' || Yes || Yes <br />
|-<br />
| WallDuration || int || || || || || <br />
|-<br />
| CpuDuration || int || || || || || <br />
|-<br />
| Processors || int || || || || || <br />
|-<br />
| NodeCount || int || || || || || <br />
|-<br />
| StartTime || datetime || No || || || || <br />
|- <br />
| EndTime || datetime || || MUL || || ||<br />
|-<br />
| InfrastructureDescription || varchar(255) || || || || || <br />
|-<br />
| InfrastructureType || varchar(255) || || || || ||<br />
|-<br />
| MemoryReal || int || || || || || <br />
|-<br />
| MemoryVirtual || int || || || || || <br />
|-<br />
| ServiceLevelType || varchar(50) || No || || || || <br />
|-<br />
| ServiceLevel || double || No || || || || <br />
|-<br />
|}<br />
<br />
== Summaries Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|-<br />
| GlobalUserName || varchar(255) || No || MUL || 'None' || Yes <br />
|- <br />
| VO || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VOGroup || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| VORole || varchar(255) || No || MUL || 'None' || Yes <br />
|-<br />
| SubmitHost || varchar(255) || Yes || || ||<br />
|-<br />
| InfrastructureType || varchar(255) || Yes || || ||<br />
|-<br />
| ServiceLevelType || varchar(50) || No || MUL || 'None' || Yes <br />
|-<br />
| ServiceLevel || double || No || MUL || 'None' || Yes <br />
|-<br />
| NodeCount || int || No || || 0 || <br />
|-<br />
| Processors || int || No || || 0 || <br />
|-<br />
| EarliestEndTime || datetime || || || || <br />
|-<br />
| LatestEndTime || datetime || || || || <br />
|-<br />
| WallDuration || int || No || || || <br />
|-<br />
| CpuDuration || int || No || || || <br />
|-<br />
| NormalisedWallDuration || int || No || || || <br />
|-<br />
| NormalisedCpuDuration || int || No || || || <br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
|}<br />
<br />
== Sync Table ==<br />
{| cellspacing="1" cellpadding="1" border="1" width="400"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
! scope="col" | Normalised<br />
|- <br />
| Site || varchar(255) || No || MUL || || Yes <br />
|-<br />
| SubmitHost || varchar(255) || || || ||<br />
|-<br />
| NumberOfJobs || int || No || || || <br />
|-<br />
| Month || int || No || MUL || || <br />
|-<br />
| Year || int || No || MUL || || <br />
|}<br />
<br />
== HepSpecHistory Table ==<br />
<br />
{| cellspacing="1" cellpadding="1" border="1" width="500"<br />
! scope="col" | ColumnName <br />
! scope="col" | Type <br />
! scope="col" | Null <br />
! scope="col" | Key<br />
! scope="col" | Default<br />
|-<br />
| UpdateTime || TimeStamp || Auto || || || <br />
|- <br />
| Site || varchar(255) || No || PRI || || <br />
|-<br />
| HepSpec06 || int || No || PRI || ||<br />
|-<br />
| NumberOfJobs || int || No || || 'None' || <br />
|-<br />
| Month || int || No || PRI || || <br />
|- <br />
| Year || int || No || PRI || || <br />
|-<br />
| EarliestEndDate || date || No || || 'None' ||<br />
|-<br />
| LatestEndDate || date || No || || 'None' || <br />
|-<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/MessagingProtocol&diff=51918APEL/MessagingProtocol2013-02-20T12:06:08Z<p>Wrogers: /* Proposed protocol */</p>
<hr />
<div>[[Category:Accounting]]<br />
= EMI Messaging Protocol for Accounting (EMPA) =<br />
<br />
Discussion about this protocol will take place on the mailing list emi-jra1-messaging@eu-emi.eu. The page for the mailing list is here: http://mail.eu-emi.eu/mailman/listinfo/emi-jra1-messaging. If you subscribe, you can also view the archives. This discussion took place from June to August 2012. I have now added version 1.0 of this protocol to the bottom of this page.<br />
<br />
== Background ==<br />
It has been agreed within EMI and EGI to use messaging for transporting accounting data. The APEL team developed software (called SSM) for use with messaging, and although it can be used independently of the message format, it uses a somewhat complex message sequence on top of the STOMP protocol, so it is not easy for other software to interact.<br />
<br />
We will discuss ways to simplify the protocol so that it is simple for different accounting software to interact.<br />
<br />
We won't consider the message ''content'' here - that is covered elsewhere, such as:<br />
* the Compute Accounting Record (CAR): see https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting <br />
* the Storage Accounting Record (StAR): see https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
<br />
<br />
== The current protocol ==<br />
<br />
[https://wiki.egi.eu/w/images/8/85/SSM_Messaging_Protocol.pdf This document] describes the current protocol used by SSM. It may not be entirely well-defined, because it was written after the software.<br />
<br />
Please feel free to comment if this document is unclear or if you think it is incorrect, and I will improve it.<br />
<br />
== Criticisms of the protocol ==<br />
<br />
[https://wiki.egi.eu/w/images/a/a2/Accounting_Messaging_Protocol_Improvements.pdf This document] attempts to describe problems with the current protocol and suggestions for improving it.<br />
<br />
Please reply with comments about the listed problems and proposals, so that we can make sure any possible issues are addressed properly.<br />
<br />
== Conclusions ==<br />
<br />
We have discussed the possibilities on the mailing list emi-jra1-messaging@eu-emi.eu, and the emails can be found in the archives for June 2012. The following main points were established:<br />
* It is useful for clients to sign the messages they send with their certificate<br />
* We should make it optional whether clients encrypt messages, and use MIME types to determine whether this encryption has taken place<br />
* Signatures and encryption should be done according to the SMIME specification<br />
* Accounting message queues should be persistent, so that consumers are guaranteed delivery. How this is implemented should depend on the details of the broker network<br />
* The 'ack' mechanism in the 'current protocol' listed above will not be retained<br />
* The certificate request and response messages in the current protocol will not be retained. If a client is to encrypt messages with the server certificate, that certificate should be distributed by another method<br />
<br />
== Proposed protocol ==<br />
<br />
This is version 1.1 of the protocol specification.<br />
<br />
[https://wiki.egi.eu/wiki/File:EMI_Accounting_Messaging_Protocol_v1.1.pdf EMI_Accounting_Messaging_Protocol_v1.1.pdf]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/MessagingProtocol&diff=51917APEL/MessagingProtocol2013-02-20T12:05:48Z<p>Wrogers: /* Proposed protocol */</p>
<hr />
<div>[[Category:Accounting]]<br />
= EMI Messaging Protocol for Accounting (EMPA) =<br />
<br />
Discussion about this protocol will take place on the mailing list emi-jra1-messaging@eu-emi.eu. The page for the mailing list is here: http://mail.eu-emi.eu/mailman/listinfo/emi-jra1-messaging. If you subscribe, you can also view the archives. This discussion took place from June to August 2012. I have now added version 1.0 of this protocol to the bottom of this page.<br />
<br />
== Background ==<br />
It has been agreed within EMI and EGI to use messaging for transporting accounting data. The APEL team developed software (called SSM) for use with messaging, and although it can be used independently of the message format, it uses a somewhat complex message sequence on top of the STOMP protocol, so it is not easy for other software to interact.<br />
<br />
We will discuss ways to simplify the protocol so that it is simple for different accounting software to interact.<br />
<br />
We won't consider the message ''content'' here - that is covered elsewhere, such as:<br />
* the Compute Accounting Record (CAR): see https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting <br />
* the Storage Accounting Record (StAR): see https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
<br />
<br />
== The current protocol ==<br />
<br />
[https://wiki.egi.eu/w/images/8/85/SSM_Messaging_Protocol.pdf This document] describes the current protocol used by SSM. It may not be entirely well-defined, because it was written after the software.<br />
<br />
Please feel free to comment if this document is unclear or if you think it is incorrect, and I will improve it.<br />
<br />
== Criticisms of the protocol ==<br />
<br />
[https://wiki.egi.eu/w/images/a/a2/Accounting_Messaging_Protocol_Improvements.pdf This document] attempts to describe problems with the current protocol and suggestions for improving it.<br />
<br />
Please reply with comments about the listed problems and proposals, so that we can make sure any possible issues are addressed properly.<br />
<br />
== Conclusions ==<br />
<br />
We have discussed the possibilities on the mailing list emi-jra1-messaging@eu-emi.eu, and the emails can be found in the archives for June 2012. The following main points were established:<br />
* It is useful for clients to sign the messages they send with their certificate<br />
* We should make it optional whether clients encrypt messages, and use MIME types to determine whether this encryption has taken place<br />
* Signatures and encryption should be done according to the SMIME specification<br />
* Accounting message queues should be persistent, so that consumers are guaranteed delivery. How this is implemented should depend on the details of the broker network<br />
* The 'ack' mechanism in the 'current protocol' listed above will not be retained<br />
* The certificate request and response messages in the current protocol will not be retained. If a client is to encrypt messages with the server certificate, that certificate should be distributed by another method<br />
<br />
== Proposed protocol ==<br />
<br />
This is version 1.1 of the protocol specification.<br />
[https://wiki.egi.eu/wiki/File:EMI_Accounting_Messaging_Protocol_v1.1.pdf EMI_Accounting_Messaging_Protocol_v1.1.pdf]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=File:EMI_Accounting_Messaging_Protocol_v1.1.pdf&diff=51916File:EMI Accounting Messaging Protocol v1.1.pdf2013-02-20T12:04:57Z<p>Wrogers: </p>
<hr />
<div></div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Configuration&diff=50999APEL/SSM2Configuration2013-02-06T16:17:25Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Configuring SSM 2 =<br />
<br />
For messages to be accepted by the APEL server, '''you must have provided the DN of your host certificate to the APEL team: email apel-ssm-test@jiscmail.ac.uk'''<br />
<br />
== CPU Accounting ==<br />
<br />
This is currently being tested, if you want to send CAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [broker] section, instead of:<br />
<code>network: PROD</code><br />
you need:<br />
<br />
<code>network: TEST-NWOB</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cputest.CENTRAL</code><br />
<br />
== Storage Accounting ==<br />
<br />
This is currently being tested, if you want to send StAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [broker] section, instead of:<br />
<code>network: PROD</code><br />
you need:<br />
<br />
<code>network: TEST-NWOB</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.storagetest.CENTRAL</code><br />
<br />
== Cloud Accounting ==<br />
<br />
This is currently being tested, if you want to send Cloud Accounting records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [broker] section, instead of:<br />
<code>network: PROD</code><br />
you need:<br />
<br />
<code>network: TEST-NWOB</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Installation|Installing SSM2]]<br />
* '''Next:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Configuration&diff=50998APEL/SSM2Configuration2013-02-06T16:16:50Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Configuring SSM 2 =<br />
<br />
For messages to be accepted by the APEL server, '''you must have provided the DN of your host certificate to the APEL team: email apel-ssm-test@jiscmail.ac.uk'''<br />
<br />
== CPU Accounting ==<br />
<br />
This is currently being tested, if you want to send CAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [broker] section, instead of:<br />
<code>network: PROD</code><br />
you need:<br />
<br />
<code>network: TEST-NWOB</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cputest.CENTRAL</code><br />
<br />
== Storage Accounting ==<br />
<br />
This is currently being tested, if you want to send StAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.storagetest.CENTRAL</code><br />
<br />
== Cloud Accounting ==<br />
<br />
This is currently being tested, if you want to send Cloud Accounting records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Installation|Installing SSM2]]<br />
* '''Next:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=50485APEL/Test APEL Server2013-01-28T12:36:38Z<p>Wrogers: /* Test APEL Server */</p>
<hr />
<div>[[Category:Accounting]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk. '''You need to provide the DN of the certificate used to the sign the messages in order for the messages to be accepted.'''<br />
<br />
== Connection details ==<br />
<br />
=== Broker ===<br />
You may connect using the following broker:<br />
* <code>stomp://test-msg02.afroditi.hellasgrid.gr:6163</code><br />
<br />
=== Queue ===<br />
===== CPU Accounting =====<br />
* <code>/queue/global.accounting.cputest.CENTRAL</code><br />
===== Storage Accounting =====<br />
* <code>/queue/global.accounting.storagetest.CENTRAL</code><br />
===== Cloud Accounting =====<br />
* <code>/queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=50484APEL/Test APEL Server2013-01-28T12:36:03Z<p>Wrogers: /* Broker */</p>
<hr />
<div>[[Category:Accounting]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk. You need to provide the DN of the certificate used to the sign the messages in order for the messages to be accepted.<br />
<br />
== Connection details ==<br />
<br />
=== Broker ===<br />
You may connect using the following broker:<br />
* <code>stomp://test-msg02.afroditi.hellasgrid.gr:6163</code><br />
<br />
=== Queue ===<br />
===== CPU Accounting =====<br />
* <code>/queue/global.accounting.cputest.CENTRAL</code><br />
===== Storage Accounting =====<br />
* <code>/queue/global.accounting.storagetest.CENTRAL</code><br />
===== Cloud Accounting =====<br />
* <code>/queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=50482APEL/Test APEL Server2013-01-28T12:26:11Z<p>Wrogers: /* Test APEL Server */</p>
<hr />
<div>[[Category:Accounting]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk. You need to provide the DN of the certificate used to the sign the messages in order for the messages to be accepted.<br />
<br />
== Connection details ==<br />
<br />
=== Broker ===<br />
You may connect using the following broker:<br />
* <code>stomp://test-msg02.afroditi.hellasgrid.gr:6163</code><br />
* <code>stomp+ssl://test-msg02.afroditi.hellasgrid.gr:6162</code><br />
<br />
=== Queue ===<br />
===== CPU Accounting =====<br />
* <code>/queue/global.accounting.cputest.CENTRAL</code><br />
===== Storage Accounting =====<br />
* <code>/queue/global.accounting.storagetest.CENTRAL</code><br />
===== Cloud Accounting =====<br />
* <code>/queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=50481APEL/Test APEL Server2013-01-28T12:23:41Z<p>Wrogers: /* Queue */</p>
<hr />
<div>[[Category:Accounting]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk.<br />
<br />
== Connection details ==<br />
<br />
=== Broker ===<br />
You may connect using the following broker:<br />
* <code>stomp://test-msg02.afroditi.hellasgrid.gr:6163</code><br />
* <code>stomp+ssl://test-msg02.afroditi.hellasgrid.gr:6162</code><br />
<br />
=== Queue ===<br />
===== CPU Accounting =====<br />
* <code>/queue/global.accounting.cputest.CENTRAL</code><br />
===== Storage Accounting =====<br />
* <code>/queue/global.accounting.storagetest.CENTRAL</code><br />
===== Cloud Accounting =====<br />
* <code>/queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=50480APEL/Test APEL Server2013-01-28T12:23:13Z<p>Wrogers: </p>
<hr />
<div>[[Category:Accounting]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk.<br />
<br />
== Connection details ==<br />
<br />
=== Broker ===<br />
You may connect using the following broker:<br />
* <code>stomp://test-msg02.afroditi.hellasgrid.gr:6163</code><br />
* <code>stomp+ssl://test-msg02.afroditi.hellasgrid.gr:6162</code><br />
<br />
=== Queue ===<br />
==== CPU Accounting ====<br />
* <code>/queue/global.accounting.cputest.CENTRAL</code><br />
==== Storage Accounting ====<br />
* <code>/queue/global.accounting.storagetest.CENTRAL</code><br />
==== Cloud Accounting ====<br />
* <code>/queue/global.accounting.cloudtest.CENTRAL</code> <br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Configuration&diff=50127APEL/SSM2Configuration2013-01-24T09:31:20Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Configuring SSM 2 =<br />
<br />
For messages to be accepted by the APEL server, '''you must have provided the DN of your host certificate to the APEL team: email apel-ssm-test@jiscmail.ac.uk'''<br />
<br />
== CPU Accounting ==<br />
<br />
This is currently being tested, if you want to send StAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cputest.CENTRAL</code><br />
<br />
== Storage Accounting ==<br />
<br />
This is currently being tested, if you want to send StAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.storagetest.CENTRAL</code><br />
<br />
== Cloud Accounting ==<br />
<br />
This is currently being tested, if you want to send Cloud Accounting records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Installation|Installing SSM2]]<br />
* '''Next:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/APELSSMExternalTesting&diff=49984APEL/APELSSMExternalTesting2013-01-22T16:46:22Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
This page is for people who were testing the SSM version 1 package. '''If you are beginning testing now, please use SSM version 2.'''<br />
<br />
== Mailing list ==<br />
<br />
Firstly, there is a mailing list set up for discussing installation and testing: apel-ssm-test@jiscmail.ac.uk. To join the list:<br />
* email listserv@jiscmail.ac.uk with '''no subject''' and message body '''subscribe apel-ssm-test firstname lastname'''<br />
<br />
You will receive an email confirming your subscription and with instructions on how to do other things, such as view old emails.<br />
<br />
== Installation ==<br />
<br />
There are detailed instructions for installing the latest version of the SSM here: [[APEL/SSMInstallation]].<br />
<br />
== Usage ==<br />
<br />
See [[APEL/RunningSSM]].<br />
<br />
Let us know using the mailing list if you are going to send us some messages, and we can tell you whether they arrive and whether they are valid.<br />
<br />
If you would like to send us messages using the new APEL message format, we can let you know whether they are accepted by the database. The message format is described on this page: [[APEL/MessageFormat]].<br />
<br />
To monitor more closely what your SSM is doing, you can look at the ssm.log file (the default location is <code>/var/log/apel/ssm.log</code>). However, we know that this file can be a bit cryptic.<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSMInstallation&diff=49983APEL/SSMInstallation2013-01-22T16:45:02Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
SSM1 downloads are available at https://github.com/apel/ssm/downloads.<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM =<br />
<br />
* For details on the testing process, see this page: [[APEL/APELSSMExternalTesting]]<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== ssm-1.2 ==<br />
<br />
The same package is installed for CPU accounting, StAR and Cloud accounting records.<br />
<br />
Version 1.2 is the current version.<br />
<br />
'''It changes the default messages location from the previous version:'''<br />
* OLD: <code>/opt/apel/ssm/messages/</code><br />
* NEW: <code>/var/opt/apel/messages/</code><br />
<br />
Installation and removal instructions are now packaged with the SSM in the README file. If you would like to read it before you install the SSM, here is the version included with ssm-1.2: [[APEL/SSM/README12]].<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSMConfiguration]] for how to change the configuration.<br />
<br />
== Older versions ==<br />
<br />
There is now no reason to install an older version. Please install SSM 1.2 or above.<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM&diff=49982APEL/SSM2013-01-22T16:44:11Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Secure Stomp Messenger =<br />
<br />
Secure Stomp Messenger (SSM) is the messaging system used by APEL to transmit messages. It is written in Python and uses the STOMP protocol.<br />
<br />
SSM is still in development, and we would be happy to hear any comments you have about design, packaging, documentation or anything else - email '''apel-ssm-test@jiscmail.ac.uk'''.<br />
<br />
For more on testing, see [[APEL/Test_APEL_Server]].<br />
<br />
<br />
== Introduction ==<br />
SSM was designed as a way of using python and STOMP to securely and reliably send messages from APEL clients to the APEL server. It is designed to use the EGI broker network (see [[Message_brokers]]), but should be compatible with any stomp broker. <br />
<br />
It is not tied to any other APEL components, so is also being used for sending StAR and Cloud accounting records.<br />
<br />
<br />
== SSM2 ==<br />
After discussion within EMI, it was decided that SSM1's design was too complicated for interoperation with other tools. The discussion led to the definition of the EMI Messaging Protocol for Accounting (EMPA), which you can find here: [[APEL/MessagingProtocol]]. SSM2 is under development to conform with EMPA. SSM2 will be released in EMI3.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are signed and may be encrypted in transit, using x509 certificates<br />
* a receiving SSM acts as a daemon process; a sending SSM runs, sending all its messages, and quits<br />
<br />
==== Downloads ====<br />
The latest version can be downloaded from this page: http://apel.github.com/apel/. Choose the apel-ssm rpm for your OS.<br />
<br />
Alternatively, you may install using the EMI3 RC repo. See [[APEL/SSM2Installation]] for instructions.<br />
<br />
==== Documentation ====<br />
* [[APEL/SSM2Overview|SSM2 Overview]]<br />
* [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
* [[APEL/RunningSSM2|Running SSM2]]<br />
<br />
== SSM1 ==<br />
This is the original and current version of SSM.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are encrypted in transit, using IGTF X509 certificates<br />
* all messages are acknowledged by the receiving SSM, so both sides know if a message has been sent<br />
* the same program can act as a sending or receiving SSM, or both<br />
* a receiving SSM must run as a daemon process; a sending SSM can run as a daemon process or can run once, sending all its messages, and quit<br />
<br />
==== Downloads ====<br />
The packages (rpm and zip) are available for download here:<br />
* https://github.com/apel/ssm/downloads<br />
<br />
==== Documentation ====<br />
<br />
* [[APEL/SSMOverview|SSM Overview]]<br />
* [[APEL/SSMInstallation|SSM installation instructions]]<br />
* [[APEL/SSMConfiguration|Configuring the SSM]]<br />
* [[APEL/RunningSSM|Running the SSM]]<br />
<br />
==== External SSM Testing ====<br />
* [[APEL/APELSSMExternalTesting|APEL SSM Testing]]: If you are interested in testing the new STOMP interface or want to develop your own client.<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM&diff=49834APEL/SSM2013-01-21T16:29:19Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Secure Stomp Messenger =<br />
<br />
Secure Stomp Messenger (SSM) is the messaging system used by APEL to transmit messages. It is written in Python and uses the STOMP protocol.<br />
<br />
SSM is still in development, and we would be happy to hear any comments you have about design, packaging, documentation or anything else - email '''apel-ssm-test@jiscmail.ac.uk'''.<br />
<br />
For more on testing, see [[APEL/Test_APEL_Server]].<br />
<br />
<br />
== Introduction ==<br />
SSM was designed as a way of using python and STOMP to securely and reliably send messages from APEL clients to the APEL server. It is designed to use the EGI broker network (see [[Message_brokers]]), but should be compatible with any stomp broker. <br />
<br />
It is not tied to any other APEL components, so is also being used for sending StAR and Cloud accounting records.<br />
<br />
<br />
== SSM2 ==<br />
After discussion within EMI, it was decided that SSM1's design was too complicated for interoperation with other tools. The discussion led to the definition of the EMI Messaging Protocol for Accounting (EMPA), which you can find here: [[APEL/MessagingProtocol]]. SSM2 is under development to conform with EMPA. SSM2 will be released in EMI3.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are signed and may be encrypted in transit, using x509 certificates<br />
* a receiving SSM acts as a daemon process; a sending SSM runs, sending all its messages, and quits<br />
<br />
==== Downloads ====<br />
The latest version can be downloaded from this page: http://apel.github.com/apel/. Choose the apel-ssm rpm for your OS.<br />
<br />
Alternatively, you may install using the EMI3 RC repo. See [[APEL/SSM2Installation]] for instructions.<br />
<br />
==== Documentation ====<br />
* [[APEL/SSM2Overview|SSM2 Overview]]<br />
* [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
* [[APEL/RunningSSM2|Running SSM2]]<br />
<br />
== SSM1 ==<br />
This is the original and current version of SSM.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are encrypted in transit, using IGTF X509 certificates<br />
* all messages are acknowledged by the receiving SSM, so both sides know if a message has been sent<br />
* the same program can act as a sending or receiving SSM, or both<br />
* a receiving SSM must run as a daemon process; a sending SSM can run as a daemon process or can run once, sending all its messages, and quit<br />
<br />
==== Downloads ====<br />
The packages (rpm and zip) are available for download here:<br />
* https://github.com/willrogers/ssm/downloads<br />
<br />
==== Documentation ====<br />
<br />
* [[APEL/SSMOverview|SSM Overview]]<br />
* [[APEL/SSMInstallation|SSM installation instructions]]<br />
* [[APEL/SSMConfiguration|Configuring the SSM]]<br />
* [[APEL/RunningSSM|Running the SSM]]<br />
<br />
==== External SSM Testing ====<br />
* [[APEL/APELSSMExternalTesting|APEL SSM Testing]]: If you are interested in testing the new STOMP interface or want to develop your own client.<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=49833APEL/Test APEL Server2013-01-21T16:28:39Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk.<br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=49832APEL/Test APEL Server2013-01-21T16:27:12Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk.<br />
<br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:<br />
* Link to be added</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=49831APEL/Test APEL Server2013-01-21T16:22:52Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk.<br />
<br />
<br />
== Data transfer ==<br />
=== SSM2 ===<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
=== SSM2-compatible ===<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA). See this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.<br />
<br />
== Message formats ==<br />
=== CPU accounting ===<br />
CPU accounting data can be in one of two formats:<br />
* APEL format: https://wiki.egi.eu/wiki/APEL/MessageFormat<br />
* EMI CAR: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting<br />
<br />
=== Storage accounting ===<br />
This must be in the EMI StAR format (v1.2):<br />
* https://twiki.cern.ch/twiki/bin/view/EMI/StorageAccounting<br />
=== Cloud accounting ===<br />
This must be in the APEL format:</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=49830APEL/Test APEL Server2013-01-21T16:12:59Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk<br />
<br />
== SSM2 ==<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].<br />
<br />
== SSM2-compatible ==<br />
<br />
Other software must comply with the EMI Messaging Protocol for Accounting (EMPA) - see this page for more: https://wiki.egi.eu/wiki/APEL/MessagingProtocol.</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Test_APEL_Server&diff=49829APEL/Test APEL Server2013-01-21T16:10:24Z<p>Wrogers: Created page with "* Back to APEL/Server = Test APEL Server = There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data. In order ..."</p>
<hr />
<div>* Back to [[APEL/Server]]<br />
<br />
= Test APEL Server =<br />
<br />
There is now a test server available, receiving data via SSM2. It will accept CPU, storage and cloud accounting data.<br />
<br />
In order to send data to the test server, you must use SSM2 or compatible software. Please coordinate testing with the APEL team: apel-ssm-test@jiscmail.ac.uk<br />
<br />
== SSM2 ==<br />
<br />
Please see the SSM page for installing and configuring SSM2: [[APEL/SSM]].</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/Server&diff=49828APEL/Server2013-01-21T16:04:50Z<p>Wrogers: </p>
<hr />
<div>== New APEL Overview ==<br />
* Sites which do not run the APEL client will send summary records, via [[APEL/SSM|SSM]], which will be inserted in the Summaries table in the central APEL database. <br />
* The APEL clients will send Job records, via SSM, which will be inserted in the JobRecords table in the central APEL database. <br />
* Summary records will be created from the data in the JobRecords table. These JobRecords Summaries will be loaded into a SuperSummaries table (the SuperSummaries table will have the same schema as the Summaries table). <br />
* The records from external clients in the Summaries table will be inserted (replaced) into the SuperSummaries table. <br />
* The SpecInt data will also be extracted and stored in a SpecInt table.<br />
* The relevant Summary and SpecInt data will then be sent to the Accounting Portal using another instance of the SSM.<br />
<br />
== New APEL Server and Accounting Portal ==<br />
* [[APEL Server and Accounting Portal]]<br />
<br />
== APEL Server Design ==<br />
The new APEL server is written in Python, the database is MySQL. It has the following components:<br />
* [[APEL/SSM|SSM]]<br />
* Record loader<br />
* Database<br />
* Summarizer<br />
* Record publisher<br />
* Authentication<br />
* Monitoring<br />
<br />
==== Test server ====<br />
* [[APEL/Test_APEL_Server]]<br />
<br />
==== Other notes ====<br />
* [[APEL/TimesAndDates]]<br />
<br />
== SSM ==<br />
<br />
For all documentation on the SSM, see [[APEL/SSM]]<br />
<br />
Here is the document about APEL and messaging: [[File:APEL-Messaging-v2.2.pdf]]<br />
<br />
Here is the discussion about the [[APEL/MessagingProtocol]].<br />
<br />
== Record loader ==<br />
This has the task of taking the received messages and inserting the records in the database. At present there are three message types defined:<br />
* Job Record messages<br />
* Summary Record messages<br />
* Sync Record messages<br />
<br />
The header in the message defines the type of records the message will contain and the version of the format. This will enable us to identify newer versions if they are defined.<br />
<br />
This simple model allows the definition of alternative message types for different records which the record loader can then be configured to handle, for example, there will be a SpecIntHistory Record message type which is a requirement of the Accounting portal.<br />
<br />
==== APEL Message Format ====<br />
* [[APEL/MessageFormat|APEL Message Format]]<br />
<br />
== Database ==<br />
* [[APEL/DatabaseTables]]<br />
<br />
== Summarizer ==<br />
This processes the Job Records and Summary Records to create the SuperSummaries (User and CPU summaries) and SpecIntHistory tables required by the portal.<br />
<br />
== Record publisher ==<br />
This takes the information from a database and produces messages to be sent to the portal from the server using an instance of the SSM. (The Record publisher will also be used by the new version of the APEL client to create messages from the APEL client database to be sent to the APEL server.)<br />
<br />
== Authentication ==<br />
This verifies that the senders of the messages via the SSM are known to APEL.<br />
<br />
* [[APEL/UsingAuth]]<br />
<br />
== Monitoring ==<br />
Two APEL accounting tests will continue to be provided, APEL-Pub and APEL-Sync. These tests are required to publish details of every production/certified site registered in GOCDB with at least one CE defined as an "APEL" service type. These results are sent as Nagios metrics via a network of ActiveMQ brokers to be consumed by the ROC (Regional Operations Centre) Nagios instances.<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Installation&diff=49827APEL/SSM2Installation2013-01-21T15:59:50Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM2 =<br />
<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== RPM ==<br />
<br />
Installation and removal instructions are packaged with the SSM in the README file. If you would like to read it before you install the SSM, see GitHub: https://github.com/apel/ssm/blob/master/README.<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSM2Configuration]] for how to change the configuration.<br />
<br />
== EMI3 Repos ==<br />
<br />
SSM is now available in the EMI3 RC repo. You must:<br />
<br />
* enable the EPEL repository<br />
* enable the appropriate repo:<br />
** '''SL5''': http://etics-repository.cern.ch/repository/pm/volatile/repomd/name/emi_R_3_rc_sl5_64/etics-volatile-build-by-id-protect.repo<br />
** '''SL6''': http://etics-repository.cern.ch/repository/pm/volatile/repomd/name/emi_R_3_rc_sl6_64/etics-volatile-build-by-id-protect.repo<br />
<br />
* <code>yum install apel-ssm</code><br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Overview|SSM2 Overview]]<br />
* '''Next:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM&diff=49826APEL/SSM2013-01-21T15:56:29Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Secure Stomp Messenger =<br />
<br />
Secure Stomp Messenger (SSM) is the messaging system used by APEL to transmit messages. It is written in Python and uses the STOMP protocol.<br />
<br />
SSM is still in development, and we would be happy to hear any comments you have about design, packaging, documentation or anything else - email '''apel-ssm-test@jiscmail.ac.uk'''.<br />
<br />
<br />
== Introduction ==<br />
SSM was designed as a way of using python and STOMP to securely and reliably send messages from APEL clients to the APEL server. It is designed to use the EGI broker network (see [[Message_brokers]]), but should be compatible with any stomp broker. <br />
<br />
It is not tied to any other APEL components, so is also being used for sending StAR and Cloud accounting records.<br />
<br />
<br />
== SSM2 ==<br />
After discussion within EMI, it was decided that SSM1's design was too complicated for interoperation with other tools. The discussion led to the definition of the EMI Messaging Protocol for Accounting (EMPA), which you can find here: [[APEL/MessagingProtocol]]. SSM2 is under development to conform with EMPA. SSM2 will be released in EMI3.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are signed and may be encrypted in transit, using x509 certificates<br />
* a receiving SSM acts as a daemon process; a sending SSM runs, sending all its messages, and quits<br />
<br />
==== Downloads ====<br />
The latest version can be downloaded from this page: http://apel.github.com/apel/. Choose the apel-ssm rpm for your OS.<br />
<br />
Alternatively, you may install using the EMI3 RC repo. See [[APEL/SSM2Installation]] for instructions.<br />
<br />
==== Documentation ====<br />
* [[APEL/SSM2Overview|SSM2 Overview]]<br />
* [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
* [[APEL/RunningSSM2|Running SSM2]]<br />
<br />
== SSM1 ==<br />
This is the original and current version of SSM.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are encrypted in transit, using IGTF X509 certificates<br />
* all messages are acknowledged by the receiving SSM, so both sides know if a message has been sent<br />
* the same program can act as a sending or receiving SSM, or both<br />
* a receiving SSM must run as a daemon process; a sending SSM can run as a daemon process or can run once, sending all its messages, and quit<br />
<br />
==== Downloads ====<br />
The packages (rpm and zip) are available for download here:<br />
* https://github.com/willrogers/ssm/downloads<br />
<br />
==== Documentation ====<br />
<br />
* [[APEL/SSMOverview|SSM Overview]]<br />
* [[APEL/SSMInstallation|SSM installation instructions]]<br />
* [[APEL/SSMConfiguration|Configuring the SSM]]<br />
* [[APEL/RunningSSM|Running the SSM]]<br />
<br />
==== External SSM Testing ====<br />
* [[APEL/APELSSMExternalTesting|APEL SSM Testing]]: If you are interested in testing the new STOMP interface or want to develop your own client.<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM&diff=49692APEL/SSM2013-01-18T09:39:58Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Secure Stomp Messenger =<br />
<br />
Secure Stomp Messenger (SSM) is the messaging system used by APEL to transmit messages. It is written in Python and uses the STOMP protocol.<br />
<br />
SSM is still in development, and we would be happy to hear any comments you have about design, packaging, documentation or anything else - email '''apel-ssm-test@jiscmail.ac.uk'''.<br />
<br />
<br />
== Introduction ==<br />
SSM was designed as a way of using python and STOMP to securely and reliably send messages from APEL clients to the APEL server. It is designed to use the EGI broker network (see [[Message_brokers]]), but should be compatible with any stomp broker. <br />
<br />
It is not tied to any other APEL components, so is also being used for sending StAR and Cloud accounting records.<br />
<br />
<br />
== SSM2 ==<br />
After discussion within EMI, it was decided that SSM1's design was too complicated for interoperation with other tools. The discussion led to the definition of the EMI Messaging Protocol for Accounting (EMPA), which you can find here: [[APEL/MessagingProtocol]]. SSM2 is under development to conform with EMPA. SSM2 will be released in EMI3.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are signed and may be encrypted in transit, using x509 certificates<br />
* a receiving SSM acts as a daemon process; a sending SSM runs, sending all its messages, and quits<br />
<br />
==== Downloads ====<br />
The latest version can be downloaded from this page: http://apel.github.com/apel/. Choose the apel-ssm rpm for your OS.<br />
<br />
==== Documentation ====<br />
* [[APEL/SSM2Overview|SSM2 Overview]]<br />
* [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
* [[APEL/RunningSSM2|Running SSM2]]<br />
<br />
== SSM1 ==<br />
This is the original and current version of SSM.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are encrypted in transit, using IGTF X509 certificates<br />
* all messages are acknowledged by the receiving SSM, so both sides know if a message has been sent<br />
* the same program can act as a sending or receiving SSM, or both<br />
* a receiving SSM must run as a daemon process; a sending SSM can run as a daemon process or can run once, sending all its messages, and quit<br />
<br />
==== Downloads ====<br />
The packages (rpm and zip) are available for download here:<br />
* https://github.com/willrogers/ssm/downloads<br />
<br />
==== Documentation ====<br />
<br />
* [[APEL/SSMOverview|SSM Overview]]<br />
* [[APEL/SSMInstallation|SSM installation instructions]]<br />
* [[APEL/SSMConfiguration|Configuring the SSM]]<br />
* [[APEL/RunningSSM|Running the SSM]]<br />
<br />
==== External SSM Testing ====<br />
* [[APEL/APELSSMExternalTesting|APEL SSM Testing]]: If you are interested in testing the new STOMP interface or want to develop your own client.<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/RunningSSM2&diff=49686APEL/RunningSSM22013-01-17T17:13:42Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Running SSM2 =<br />
<br />
== Running SSM2 as a sender ==<br />
<br />
See [[APEL/SSM2AddingFiles]] for how to add messages to be sent.<br />
<br />
To run the sender, use this command:<br />
<br />
* <code>ssmsend</code><br />
<br />
The default configuration will log to the console and to file <code>/var/log/apel/ssmsend.log</code>. You may change this in the configuration file <code>/etc/apel/sender.cfg</code>.<br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Installation&diff=49685APEL/SSM2Installation2013-01-17T17:11:53Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM2 =<br />
<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== SSM 2 ==<br />
<br />
Installation and removal instructions are packaged with the SSM in the README file. If you would like to read it before you install the SSM, see GitHub: https://github.com/apel/ssm/blob/master/README.<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSM2Configuration]] for how to change the configuration.<br />
<br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Overview|SSM2 Overview]]<br />
* '''Next:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Installation&diff=49684APEL/SSM2Installation2013-01-17T17:11:40Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM2 =<br />
<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== SSM 2 ==<br />
<br />
Installation and removal instructions are packaged with the SSM in the README file. If you would like to read it before you install the SSM, see GitHub: https://github.com/apel/ssm/blob/master/README.<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSM2Configuration]] for how to change the configuration.<br />
<br />
<br />
<br />
----<br />
* '''Previous:''' * [[APEL/SSM2Overview|SSM2 Overview]]<br />
* '''Next:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Configuration&diff=49683APEL/SSM2Configuration2013-01-17T17:11:05Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Configuring SSM 2 =<br />
<br />
For messages to be accepted by the APEL server, '''you must have provided the DN of your host certificate to the APEL team: email apel-ssm-test@jiscmail.ac.uk'''<br />
<br />
== CPU Accounting ==<br />
<br />
This section will be added soon.<br />
<br />
== Storage Accounting ==<br />
<br />
This is currently being tested, if you want to send StAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.storagetest.CENTRAL</code><br />
<br />
== Cloud Accounting ==<br />
<br />
This is currently being tested, if you want to send Cloud Accounting records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Installation|Installing SSM2]]<br />
* '''Next:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/RunningSSM2&diff=49682APEL/RunningSSM22013-01-17T17:10:35Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Running SSM2 =<br />
<br />
== Running SSM2 as a sender ==<br />
<br />
See [[APEL/SSM2AddingFiles]] for how to add messages to be sent.<br />
<br />
To run the sender, use this command:<br />
<br />
* <code>ssmsend</code><br />
<br />
The default configuration will log to the console.<br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/RunningSSM2&diff=49681APEL/RunningSSM22013-01-17T17:10:04Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Running SSM2 =<br />
<br />
== Running SSM2 as a sender ==<br />
<br />
See [[APEL/SSM2AddingFiles]] for how to add messages to be sent.<br />
<br />
To run the sender, use this command:<br />
<br />
* <code>ssmsend</code><br />
<br />
The default configuration will log to the console.<br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2AddingFiles&diff=49680APEL/SSM2AddingFiles2013-01-17T17:08:45Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Adding files to SSM2 =<br />
<br />
In SSM2 we changed to using the EMI Messaging team's 'directory queue' libraries: https://twiki.cern.ch/twiki/bin/view/EMI/MessagingLibraries. This changes how messages are added to SSM2.<br />
<br />
There are two ways of adding messages to SSM2:<br />
* Programmatically, using perl or python <br />
* Manually, writing files to the filesystem<br />
<br />
== Programmatically ==<br />
<br />
You can add messages to SSM2 using the <code>python-dirq</code> or the <code>perl-Directory-Queue</code> libraries.<br />
<br />
Create a QueueSimple object with path <code>/var/spool/apel/outgoing</code> and add messages. Follow the above link or email apel-ssm-test@jiscmail.ac.uk for more help.<br />
<br />
== Manually ==<br />
<br />
If you are not using perl or python, you can simply write files to the filesystem and SSM2 will send them. However, there are constraints on the filenames:<br />
* Files must be in the location <code>/var/spool/apel/outgoing/<directory>/<file></code><br />
* '''<directory> must be 8 hex characters''' - that is 0-9 or a-f<br />
* '''<file> must be 14 hex characters'''<br />
<br />
==== Example naming scheme ====<br />
<br />
These notes are taken from the dirq documentation and describe a possible (not compulsory) naming scheme.<br />
<br />
<code><br />
Directory Structure<br />
-------------------<br />
<br />
The toplevel directory contains intermediate directories that contain<br />
the stored elements, each of them in a file.<br />
<br />
The names of the intermediate directories are time based: the element<br />
insertion time is used to create a 8-digits long hexadecimal number.<br />
The granularity (see the constructor) is used to limit the number of<br />
new directories. For instance, with a granularity of 60 (the default),<br />
new directories will be created at most once per minute.<br />
<br />
Since there is usually a filesystem limit in the number of directories<br />
a directory can hold, there is a trade-off to be made. If you want to<br />
support many added elements per second, you should use a low<br />
granularity to keep small directories. However, in this case, you will<br />
create many directories and this will limit the total number of<br />
elements you can store.<br />
<br />
The elements themselves are stored in files (one per element) with a<br />
14-digits long hexadecimal name SSSSSSSSMMMMMR where:<br />
<br />
* SSSSSSSS represents the number of seconds since the Epoch<br />
<br />
* MMMMM represents the microsecond part of the time since the Epoch<br />
<br />
* R is a random digit used to reduce name collisions<br />
</code><br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* '''Next:''' [[APEL/RunningSSM2|Running SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Configuration&diff=49679APEL/SSM2Configuration2013-01-17T17:07:20Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
= Configuring SSM 2 =<br />
<br />
For messages to be accepted by the APEL server, '''you must have provided the DN of your host certificate to the APEL team: email apel-ssm-test@jiscmail.ac.uk'''<br />
<br />
== CPU Accounting ==<br />
<br />
This section will be added soon.<br />
<br />
== Storage Accounting ==<br />
<br />
This is currently being tested, if you want to send StAR records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.storagetest.CENTRAL</code><br />
<br />
== Cloud Accounting ==<br />
<br />
This is currently being tested, if you want to send Cloud Accounting records to the test server you should make the following changes:<br />
<br />
Update the configuration file <code>/etc/apel/sender.cfg</code><br />
<br />
In the [messaging] section, instead of:<br />
<code>destination: /queue/ssm2test</code><br />
you need:<br />
<br />
<code>destination: /queue/global.accounting.cloudtest.CENTRAL</code><br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* '''Next:''' [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Installation&diff=49678APEL/SSM2Installation2013-01-17T17:06:10Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM2 =<br />
<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== SSM 2 ==<br />
<br />
Installation and removal instructions are packaged with the SSM in the README file. If you would like to read it before you install the SSM, see GitHub: https://github.com/apel/ssm/blob/master/README.<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSM2Configuration]] for how to change the configuration.<br />
<br />
<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* '''Next:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Installation&diff=49677APEL/SSM2Installation2013-01-17T17:06:03Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM2 =<br />
<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== SSM 2 ==<br />
<br />
Installation and removal instructions are packaged with the SSM in the README file. If you would like to read it before you install the SSM, see GitHub: https://github.com/apel/ssm/blob/master/README.<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSM2Configuration]] for how to change the configuration.<br />
<br />
----<br />
* '''Previous:''' [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* '''Next:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Installation&diff=49676APEL/SSM2Installation2013-01-17T17:05:07Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]].<br />
<br />
'''NOTE:''' Before you can send your messages to the Accounting server you must send your host certificate DN to the APEL-SUPPORT@JISCMAIL.AC.UK mailing list, e.g. output from running<br />
<br />
<code>openssl x509 -subject -noout -in hostcert.pem</code><br />
<br />
= Installing SSM2 =<br />
<br />
* If you find any bugs please let us know about them: apel-ssm-test@mailtalk.ac.uk.<br />
<br />
== SSM 2 ==<br />
<br />
Installation and removal instructions are packaged with the SSM in the README file. If you would like to read it before you install the SSM, see GitHub: https://github.com/apel/ssm/blob/master/README.<br />
<br />
'''Please note''': the default configuration sends messages to the CPU Accounting test server. See [[APEL/SSM2Configuration]] for how to change the configuration.<br />
<br />
<br />
<br />
* '''Previous:''' [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* '''Next:''' [[APEL/SSM2Configuration|Configuring SSM2]]<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Overview&diff=49675APEL/SSM2Overview2013-01-17T17:04:23Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
'''These notes refer to SSM version 2. See [[APEL/SSMOverview]] for notes on SSM1.''' Please send suggestions and queries to <br />
apel-ssm-test@jiscmail.ac.uk<br />
<br />
== Introduction ==<br />
<br />
The Secure Stomp Messenger (SSM) is a python package designed to send arbitrary files using ActiveMQ and the STOMP protocol. Its key features are:<br />
<br />
* It uses the underlying filesystem to store outgoing and / or incoming messages<br />
* It sends arbitrary files from a client to a server SSM<br />
* It can use any broker configured to use STOMP<br />
* Files are signed and may be encrypted during transit using X509 certificates<br />
<br />
== Interface ==<br />
<br />
The SSM is designed to have a simple interface, which isolates it from any other components it may be used with.<br />
<br />
==== Simplified version ====<br />
# The sending and receiving SSMs are configured to communicate using a specified queue.<br />
# A file is written to the sending SSM's 'outgoing' directory (by some other process) - see [[APEL/SSM2AddingFiles]].<br />
# The file disappears from outgoing directory when <code>ssmsend</code> is run.<br />
# The file appears in receiving SSM's 'incoming' directory if <ssmreceive> is running.<br />
<br />
<br />
Once the sending and receiving SSMs are configured and running correctly, to send a message, put a file in the correct outgoing directory with the correct name, run <code>ssmsend</code>, and it will disappear. It will arrive in the incoming directory of the receiving SSM.<br />
<br />
* '''Next:''' [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM2Overview&diff=49674APEL/SSM2Overview2013-01-17T17:04:03Z<p>Wrogers: /* Simplified version */</p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
'''These notes refer to SSM version 2. See [[APEL/SSMOverview]] for notes on SSM1.''' Please send suggestions and queries to <br />
apel-ssm-test@jiscmail.ac.uk<br />
<br />
== Introduction ==<br />
<br />
The Secure Stomp Messenger (SSM) is a python package designed to send arbitrary files using ActiveMQ and the STOMP protocol. Its key features are:<br />
<br />
* It uses the underlying filesystem to store outgoing and / or incoming messages<br />
* It sends arbitrary files from a client to a server SSM<br />
* It can use any broker configured to use STOMP<br />
* Files are signed and may be encrypted during transit using X509 certificates<br />
<br />
== Interface ==<br />
<br />
The SSM is designed to have a simple interface, which isolates it from any other components it may be used with.<br />
<br />
==== Simplified version ====<br />
# The sending and receiving SSMs are configured to communicate using a specified queue.<br />
# A file is written to the sending SSM's 'outgoing' directory (by some other process) - see [[APEL/SSM2AddingFiles]].<br />
# The file disappears from outgoing directory when <code>ssmsend</code> is run.<br />
# The file appears in receiving SSM's 'incoming' directory if <ssmreceive> is running.<br />
<br />
<br />
Once the sending and receiving SSMs are configured and running correctly, to send a message, put a file in the correct outgoing directory with the correct name, run <code>ssmsend</code>, and it will disappear. It will arrive in the incoming directory of the receiving SSM.<br />
<br />
* '''Next:'''[[APEL/SSM2Installation|SSM2 installation instructions]]<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSM&diff=49673APEL/SSM2013-01-17T17:02:33Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL]]<br />
* Back to [[APEL/Server]]<br />
<br />
= Secure Stomp Messenger =<br />
<br />
The Secure Stomp Messenger (SSM) is the messaging system used by APEL to transmit messages. It is written in Python and uses the STOMP protocol.<br />
<br />
The SSM is still in development, and we would be happy to hear any comments you have about design, packaging, documentation or anything else - email '''apel-ssm-test@jiscmail.ac.uk'''.<br />
<br />
<br />
== Introduction ==<br />
The SSM was designed as a way of using python and STOMP to securely and reliably send messages from APEL clients to the APEL server. It is designed to use the EGI broker network (see [[Message_brokers]]), but should be compatible with any stomp broker. <br />
<br />
It is not tied to any other APEL components, so is also being used for sending StAR and Cloud accounting records.<br />
<br />
<br />
== SSM2 ==<br />
After discussion within EMI, it was decided that SSM1's design was too complicated for interoperation with other tools. The discussion led to the definition of the EMI Messaging Protocol for Accounting (EMPA), which you can find here: [[APEL/MessagingProtocol]]. SSM2 is under development to conform with EMPA. SSM2 will be released in EMI3.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are signed and may be encrypted in transit, using x509 certificates<br />
* a receiving SSM acts as a daemon process; a sending SSM runs, sending all its messages, and quits<br />
<br />
==== Downloads ====<br />
The latest version can be downloaded from this page: http://apel.github.com/apel/. Choose the apel-ssm rpm for your OS.<br />
<br />
==== Documentation ====<br />
* [[APEL/SSM2Overview|SSM2 Overview]]<br />
* [[APEL/SSM2Installation|SSM2 installation instructions]]<br />
* [[APEL/SSM2Configuration|Configuring SSM2]]<br />
* [[APEL/SSM2AddingFiles|Adding files to SSM2]]<br />
* [[APEL/RunningSSM2|Running SSM2]]<br />
<br />
== SSM1 ==<br />
This is the original and current version of SSM.<br />
<br />
Key features:<br />
* any file can be sent as a message<br />
* messages are encrypted in transit, using IGTF X509 certificates<br />
* all messages are acknowledged by the receiving SSM, so both sides know if a message has been sent<br />
* the same program can act as a sending or receiving SSM, or both<br />
* a receiving SSM must run as a daemon process; a sending SSM can run as a daemon process or can run once, sending all its messages, and quit<br />
<br />
==== Downloads ====<br />
The packages (rpm and zip) are available for download here:<br />
* https://github.com/willrogers/ssm/downloads<br />
<br />
==== Documentation ====<br />
<br />
* [[APEL/SSMOverview|SSM Overview]]<br />
* [[APEL/SSMInstallation|SSM installation instructions]]<br />
* [[APEL/SSMConfiguration|Configuring the SSM]]<br />
* [[APEL/RunningSSM|Running the SSM]]<br />
<br />
==== External SSM Testing ====<br />
* [[APEL/APELSSMExternalTesting|APEL SSM Testing]]: If you are interested in testing the new STOMP interface or want to develop your own client.<br />
<br />
<br />
[[Category:Accounting]]</div>Wrogershttps://wiki.egi.eu/w/index.php?title=APEL/SSMOverview&diff=49672APEL/SSMOverview2013-01-17T17:01:12Z<p>Wrogers: </p>
<hr />
<div>* Back to [[APEL/SSM]]<br />
<br />
'''These notes refer to SSM version 1. See [[APEL/SSM2Overview]] for notes on SSM2.''' Please send suggestions and queries to apel-admins@mailtalk.ac.uk.<br />
<br />
== Introduction ==<br />
<br />
The Secure Stomp Messenger (SSM) is a python package designed to send arbitrary files using ActiveMQ and the STOMP protocol. Its key features are:<br />
<br />
* It uses the underlying filesystem to store outgoing and / or incoming messages<br />
* It sends arbitrary files from a client to a server SSM<br />
* It can use any broker configured to use STOMP<br />
* Files are encrypted during transit using X509 certificates<br />
* Files are sent sequentially - the next file is sent only when receipt of the previous file is confirmed<br />
<br />
== Interface ==<br />
<br />
The SSM is designed to have a simple interface, which isolates it from any other components it may be used with.<br />
<br />
==== Very simplified version ====<br />
# The sending and receiving SSMs are configured to communicate using a specified topic.<br />
# A file is written to the sending SSM's 'outgoing' directory (by some other process)<br />
# The file disappears from outgoing directory.<br />
# The file appears in receiving SSM's 'incoming' directory along with a second file containing the sending SSM's certificate DN.<br />
<br />
==== Less simplified version ====<br />
<br />
* The sending SSM:<br />
** requests the receiving SSM's certificate<br />
** takes files from a directory on its filesystem<br />
** encrypts (using requested certificate) and signs (using own key) the files<br />
** sends the encrypted files to a specified ActiveMQ topic.<br />
* The receiving SSM:<br />
** receives the message from the topic<br />
** decrypts the message (using own key)<br />
** retrieves the sender's certificate (from signature)<br />
** stores the message and the sender's certificate DN (in separate files) in a directory on its filesystem.<br />
<br />
Once the sending and receiving SSMs are configured and running correctly, to send a message, put a file in the correct outgoing directory and it will disappear. It will promptly appear in the incoming directory of the receiving SSM.<br />
<br />
[[Category:Accounting]]</div>Wrogers