Difference between revisions of "APEL/SSM2AddingFiles"
m |
imported>Acatstfc m |
(No difference)
|
Revision as of 18:42, 1 November 2018
- Back to APEL/SSM
Adding files to SSM2
In SSM2 we changed to using the EMI Messaging team's 'directory queue' libraries: https://twiki.cern.ch/twiki/bin/view/EMI/MessagingLibraries. This changes how messages are added to SSM2.
There are two ways of adding messages to SSM2:
- Programmatically, using perl or python
- Manually, writing files to the filesystem
Programmatically
You can add messages to SSM2 using the python-dirq
or the perl-Directory-Queue
libraries.
Create a QueueSimple object with path /var/spool/apel/outgoing
and add messages. Follow the above link or email apel-admins@stfc.ac.uk for more help.
Manually
If you are not using perl or python, you can simply write files to the filesystem and SSM2 will send them. However, there are constraints on the filenames:
- Files must be in the location
/var/spool/apel/outgoing/<directory>/<file>
- <directory> must be 8 hex characters - that is 0-9 or a-f
- <file> must be 14 hex characters
Example naming scheme
These notes are taken from the dirq documentation and describe a possible (not compulsory) naming scheme.
Directory Structure
The toplevel directory contains intermediate directories that contain the stored elements, each of them in a file.
The names of the intermediate directories are time based: the element insertion time is used to create a 8-digits long hexadecimal number. The granularity (see the constructor) is used to limit the number of new directories. For instance, with a granularity of 60 (the default), new directories will be created at most once per minute.
Since there is usually a filesystem limit in the number of directories a directory can hold, there is a trade-off to be made. If you want to support many added elements per second, you should use a low granularity to keep small directories. However, in this case, you will create many directories and this will limit the total number of elements you can store.
The elements themselves are stored in files (one per element) with a 14-digits long hexadecimal name SSSSSSSSMMMMMR where:
- SSSSSSSS represents the number of seconds since the Epoch
- MMMMM represents the microsecond part of the time since the Epoch
- R is a random digit used to reduce name collisions
- Previous: Configuring SSM2
- Next: Running SSM2