Keys in the ODB /Logger/Channels subtree: Difference between revisions

From MidasWiki
Jump to navigation Jump to search
No edit summary
mNo edit summary
 
(7 intermediate revisions by the same user not shown)
Line 3: Line 3:
* [[Keys in the ODB /Logger tree| /Logger tree]]  
* [[Keys in the ODB /Logger tree| /Logger tree]]  
* [[Keys in the ODB /Logger/SQL subtree |/Logger/SQL subtree]]
* [[Keys in the ODB /Logger/SQL subtree |/Logger/SQL subtree]]
* [[/Logger ODB tree #Example | example tree]]
* [[/Logger ODB tree #Example | Example /Logger tree]]
* [[mlogger]]  
* [[mlogger]]  
</div>
</div>
Line 39: Line 39:
* <span style="color: purple;">''...</span>
* <span style="color: purple;">''...</span>
present, each with a  
present, each with a  
<span style="color: purple;">''Settings''</span> and <span style="color: purple;">''Statistics''</span> subtree.  
[[#Settings subtree|Settings]] and [[#Statistics subtree|Statistics]] subtree.  


Thus '''the parameters for each logging channel can be set individually.'''
Thus '''the parameters for each logging channel can be set individually.'''


To send different events to different logging channels, they may be selected by [[#/Logger/Channels/0/Settings/Event ID | Event ID key]] or  
To send different events to different logging channels, they may be selected by [[#Event ID | Event ID key]] or  
[[#/Logger/Channels/0/Settings/Trigger mask | Trigger mask key]].  
[[#Trigger mask | Trigger mask key]].  


<br>
<br>
Line 50: Line 50:
<br>
<br>


== <span style="color: purple;">''/Logger/Channels/<logging-channel-number>/Settings''</span> subtree ==
== <span style="color: purple;">''Settings''</span> subtree ==


The keys in the ODB <span style="color: purple;">''/Logger/Channels/<logging-channel-number>/Settings''</span> subtree are shown below
The keys in the ODB <span style="color: purple;">''/Logger/Channels/<logging-channel-number>/Settings''</span> subtree are shown below
Line 71: Line 71:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Active''</span> ===
=== <span style="color: purple;">''Active''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' BOOL
* '''Type:''' BOOL
Line 77: Line 77:
</div>
</div>


If set to "y", this logging channel is active. Data are only logged from channels that are active.
If this ODB key in the  [[#Settings subtree|Settings subtree]] is set to "y", this logging channel is active. Data are only logged from channels that are active.
Set to "n" to disable data logging from this channel.
Set to "n" to disable data logging from this channel.


Line 87: Line 87:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Type''</span> ===
=== <span style="color: purple;">''Type''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Type:''' STRING
Line 93: Line 93:
</div>  
</div>  


Specify the type of media on which the logging should take place. It can be "Disk", "Tape" or "FTP" (see [[#/Logger/Channels/0/Settings/Filename |Filename key]]).
This ODB key  in the [[#Settings subtree|Settings subtree]] specifies the type of media on which the logging should take place. It can be "Disk", "Tape" or "FTP" (see [[#/Logger/Channels/0/Settings/Filename |Filename key]]).


<br>
<br>
Line 99: Line 99:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Filename''</span> ===
=== <span style="color: purple;">''Filename''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Type:''' STRING
Line 105: Line 105:
</div>  
</div>  


; For '''disk''' logging : Key <span style="color: purple;">''Filename''</span> specifies the name of a file to contain the data. Default value is "run%05d.mid" where "%05d" is replaced by the current run number in the same way as described for the ODB key [[Keys in the ODB /Logger tree #/Logger/Dump File |/Logger/Dump File]]. If subruns are enabled, the filename may be of the form "run%05d_%02d.mid". See [[subruns]] for details.
This ODB key in the [[#Settings subtree|Settings subtree]] is set up as follows:
 
; For '''disk''' logging : ODB key <span style="color: purple;">''Filename''</span> specifies the name of a file to contain the data. Default value is "run%05d.mid" where "%05d" is replaced by the current run number in the same way as described for the ODB key [[Keys in the ODB /Logger tree #/Logger/Dump File |/Logger/Dump File]]. If subruns are enabled, the filename may be of the form "run%05d_%02d.mid". See [[subruns]] for details.
; : The key <span style="color: purple;">''Filename''</span> may contain a directory string in addition to a filename, so that different logging channels may have their data logged into specific directories. If the key does NOT also contain a directory string, the directory given in the key [[Keys in the ODB /Logger tree #/Logger/Data dir|/Logger/Data dir key]] will be used. The  contents of the key <span style="color: purple;">''Filename''</span> must match that of the [[#/Logger/Channels/0/Settings/Format |Format key]].
; : The key <span style="color: purple;">''Filename''</span> may contain a directory string in addition to a filename, so that different logging channels may have their data logged into specific directories. If the key does NOT also contain a directory string, the directory given in the key [[Keys in the ODB /Logger tree #/Logger/Data dir|/Logger/Data dir key]] will be used. The  contents of the key <span style="color: purple;">''Filename''</span> must match that of the [[#/Logger/Channels/0/Settings/Format |Format key]].


Line 113: Line 115:
       <li>  \\.\tape0 under Windows NT. </li>
       <li>  \\.\tape0 under Windows NT. </li>
</ul>
</ul>
<div style="background-color:mistyrose">WARNING: tape logging is no longer maintained</div>


; FTP mode :  the data is written directly to a remote computer via FTP. Key <span style="color: purple;">''Filename''</span> specifies the access information for the FTP server. It has the following format:
; FTP mode :  the data is written directly to a remote computer via FTP. Key <span style="color: purple;">''Filename''</span> specifies the access information for the FTP server. It has the following format:
Line 130: Line 133:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Format''</span> ===
=== <span style="color: purple;">''Format''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Type:''' STRING
Line 136: Line 139:
</div>  
</div>  


Specifies the format to be used for writing the data to the logging channel. It can take one of four values: MIDAS, ROOT, ASCII and DUMP.
This ODB key  in the [[#Settings subtree|Settings subtree]] specifies the format to be used for writing the data to the logging channel. It can take one of four values: MIDAS, ROOT, ASCII and DUMP.


For disk logging, the extension for the Filename Key above must match one of the following:
For disk logging, the extension for the [[#Filename|Filename]] key above must match one of the following:
<ul>
<ul>
<li> .mid for MIDAS (or .mid.gz - see [[#/Logger/Channels/0/Settings/Compression | Compression key]])</li>
<li> .mid for MIDAS (or .mid.gz - see [[#Compression | Compression]] key)</li>
<li> .root for ROOT</li>
<li> .root for ROOT</li>
<li> .asc for ASCII</li>
<li> .asc for ASCII</li>
Line 146: Line 149:
</ul>
</ul>


For [[ROOT]] format the data logger will automatically convert the format specified in the [[frontend]] equipment list to ROOT format (see format for details). The ASCII format converts events into readable text format which can be easily analyzed by programs that have problems reading binary data. While the ASCII format tries to minimize the file size by printing one event per line, the DUMP format gives a very detailed ASCII representation of the event including bank information, serial numbers etc., so it should be used mainly for diagnostics.
For [[ROOT]] format the data logger will automatically convert the format specified in the [[Explanation of Equipment List Parameters#Format|frontend equipment list]] to ROOT format (see format for details). The ASCII format converts events into readable text format which can be easily analyzed by programs that have problems reading binary data. While the ASCII format tries to minimize the file size by printing one event per line, the DUMP format gives a very detailed ASCII representation of the event including bank information, serial numbers etc., so it should be used mainly for diagnostics.


<br>
<br>
Line 152: Line 155:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Compression''</span> ===
=== <span style="color: purple;">''Compression''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' INT
* '''Type:''' INT
Line 159: Line 162:


The key <span style="color: purple;">''Compression''</span> defines the Compression Level of the data. If set to 0, compression is turned off. If set > 0, the data files are compressed before being written. In this case,
This ODB key in the [[#Settings subtree|Settings subtree]] defines the Compression Level of the data. If set to 0, compression is turned off. If set > 0, the data files are compressed before being written. In this case,


*  the file extension ".gz" must be appended to the [[#/Logger/Channels/0/Settings/Filename |Filename key]] (see also  [[#/Logger/Channels/0/Settings/Format |Format key]])
*  the file extension ".gz" must be appended to the [[#/Logger/Channels/0/Settings/Filename |Filename key]] (see also  [[#Format |Format]] key)
*  and ZLIB support must be compiled into the MIDAS package (see [[NEED_ZLIB]] ).
*  and ZLIB support must be compiled into the MIDAS package (see [[Compilation & Build #NEED_ZLIB |NEED_ZLIB]] ).


This feature applies only to files saved onto disk in MIDAS format. ROOT format is already compressed.
This feature applies only to files saved onto disk in [[Event Structure|MIDAS format]]. ROOT format is already compressed.


<br>
<br>
Line 170: Line 173:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/ODB Dump''</span> ===
=== <span style="color: purple;">''ODB Dump''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' BOOL
* '''Type:''' BOOL
Line 177: Line 180:




If this flag is set, a complete dump of the ODB is performed to this logging channel before and after every run. The ODB content is dumped in one long ASCII string reflecting the status at begin-of-run event and at end-of-run event. These special events have an ID of [[EVENTID_BOR]] and [[EVENTID_EOR]] and a serial number equal to the current run number, and are not visible using the [[mdump]] utility. An analyzer in the off-line analysis stage can use these special events to restore the ODB to its online state. The dump of the ODB into the event stream for this logger channel is independent of the [[Keys in the ODB /Logger tree #/Logger/ODB Dump  |/Logger/ODB Dump key]], which is used to dump the ODB to an ASCII or XML file on disk.
If this ODB key in the [[#Settings subtree|Settings subtree]] is set to "y", a complete dump of the ODB is performed to this logging channel before and after every run.  The format of the ODB dump is set by the key [[#ODB dump format]]. The ODB content is reflects the status at begin-of-run event and at end-of-run event. These special events have an ID of EVENTID_BOR (0x8000) and EVENTID_EOR (0x8001) (defined in 
[http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/midas_8h.html midas.h]) and a serial number equal to the current run number, and are not visible using the [[mdump]] utility. An analyzer in the off-line analysis stage can use these special events to restore the ODB to its online state.  
 
The dump of the ODB into the event stream for this logger channel is independent of the [[Keys in the ODB /Logger tree #/Logger/ODB Dump  |/Logger/ODB Dump key]], which is used to dump the ODB to a file on disk in the [[#ODB dump format|specified format]].


<br>
<br>
Line 183: Line 189:
<br>
<br>


=== <span style="color: purple;">''ODB dump format''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:'''  "json"
</div>
This ODB key in the [[#Settings subtree|Settings subtree]] defines the format of the ODB Dump file, providing
the ODB key [[#ODB Dump]] is set to "y". The default is now "json" (since March 2019) and the filename of the odb dump file
saved at end-of-run is  is "last.json". 


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Log messages''</span> ===
Users who wish to use "xml" should set this key to "xml" and [[Keys in the ODB /Logger tree#ODB Last Dump File]] to "last.xml".
<br>
-----
<br>
=== <span style="color: purple;">''Log messages''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' DWORD
* '''Type:''' DWORD
Line 190: Line 210:
</div>  
</div>  


This is a bit-field for logging system messages as message events along with the event data from the experiment.
This ODB key in the [[#Settings subtree|Settings subtree]] is a bit-field for logging system messages as message events along with the event data from the experiment.
If a bit in this field is set, the respective system message is written to the logging channel as a message event with an ID of [[EVENTID_MESSAGE]] (0x8002). The bits are 1 for error, 2 for info, 4 for debug, 8 for user, 16 for log, 32 for talk, 64 for call messages and 255 to log all messages. For an explanation of these messages refer to [[Buffer Manager]] ,  
If a bit in this field is set, the respective system message is written to the logging channel as a message event with an ID of EVENTID_MESSAGE (0x8002) (defined in 
[[#/Logger/Channels/0/Settings/Event ID | Event ID key]] and  
[http://ladd00.triumf.ca/~daqweb/doc/midas-devel/doc/html/midas_8h.html midas.h]). The bits are 1 for error, 2 for info, 4 for debug, 8 for user, 16 for log, 32 for talk, 64 for call messages and 255 to log all messages. For an explanation of these messages refer to [[Buffer Manager]] ,  
[[#/Logger/Channels/0/Settings/Trigger mask | Trigger mask key]].  
[[#Event ID | Event ID]] key and  
[[#Trigger mask | Trigger mask]] key.  




Line 201: Line 222:




=== <span style="color: purple;">''/Logger/Channels/0/Settings/Buffer''</span> ===
=== <span style="color: purple;">''Buffer''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Type:''' STRING
Line 208: Line 229:




This specifies the buffer where the logger will find the event. It is usually set to "SYSTEM" and is specified in the equipment list for this event.
This ODB key in the [[#Settings subtree|Settings subtree]]  specifies the buffer where the logger will find the event. It is usually set to "SYSTEM" and is specified in the equipment list for this event.


<br>
<br>
Line 216: Line 237:
<div id="EventFiltering"></div>
<div id="EventFiltering"></div>
<div id="EventID"></div>
<div id="EventID"></div>
=== <span style="color: purple;">''/Logger/Channels/0/Settings/Event ID ''</span> ===
=== <span style="color: purple;">''Event ID ''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' INT
* '''Type:''' INT
Line 223: Line 244:




This is the ID of the event(s) to be logged (see [[MIDAS Event Structure|EventID]]). It is specified in the [[Equipment List Parameters#EventID|Equipment List]] for this event.
This ODB key in the [[#Settings subtree|Settings subtree]] contains the ID of the event(s) to be logged (see [[MIDAS Event Structure|EventID]]). It is specified in the [[Equipment List Parameters#EventID|Equipment List]] for this event.


To log events with any ID, the Event ID is set to -1 (EVENTID_ALL). The Event ID parameter and/or the Trigger Mask (see also [[#/Logger/Channels/0/Settings/Trigger mask | Trigger mask key]]) may be used to select events to be sent to different logging channels, or as an [[Event Filtering]] system.
To log events with any ID, <span style="color: purple; font-style:italic;">Event ID</span> is set to -1 (EVENTID_ALL). <span style="color: purple; font-style:italic;">Event ID</span> key and/or [[#/Logger/Channels/0/Settings/Trigger mask | Trigger mask ]] key may be used to select events to be sent to different logging channels, or as an [[Event Filtering]] system.


<br>
<br>
Line 231: Line 252:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Trigger mask''</span> ===
=== <span style="color: purple;">''Trigger mask''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' INT
* '''Type:''' INT
Line 237: Line 258:
</div>  
</div>  


This key specifies which events to log based on the [[MIDAS Event Structure|Trigger Mask]] of the event. It is specified in the [[Equipment List Parameters#TriggerMask|Equipment List]] of this event.  
This ODB key in the [[#Settings subtree|Settings subtree]] specifies which events to log based on the [[MIDAS Event Structure|Trigger Mask]] of the event. It is specified in the [[Equipment List Parameters#TriggerMask|Equipment List]] of this event.  


For a logging channel to receive all events, -1 is used for both the  
For a logging channel to receive all events, -1 is used for both the  
Line 243: Line 264:
and the <span style="color: purple;">''Trigger mask''</span> key .
and the <span style="color: purple;">''Trigger mask''</span> key .


Either Trigger Mask or  Event ID may be used to select events to be sent to particular logging channels, or for [[Event Filtering]].
Either the <span style="color: purple; font-style:italic;">Trigger Mask</span> or  <span style="color: purple; font-style:italic;">Event ID</span> key may be used to select events to be sent to particular logging channels, or for [[Event Filtering]].


<br>
<br>
Line 249: Line 270:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Event limit''</span> ===
=== <span style="color: purple;">''Event limit''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' DOUBLE
* '''Type:''' DOUBLE
Line 256: Line 277:




This field can be used to stop a run when set to a non-zero value. The statistics value Events Written is checked against the limit Event limit . When this condition is reached, the run is stopped automatically by the logger. The next run can then be started automatically if desired (see Auto restart ).
The  value of this ODB key <span style="color: purple; font-style:italic;">Event limit</span>  in the [[#Settings subtree|Settings subtree]]  can be used to stop a run when set to a non-zero value. The statistics value [[#Events written|Events written]] is checked against the <span style="color: purple; font-style:italic;">Event limit</span> value. When this condition is reached, the run is stopped automatically by the data logger [[Mlogger|mlogger]]. The next run can then be started automatically if desired (see [[Keys in the ODB /Logger tree #/Logger/Auto restart|Auto restart]]).


<br>
<br>
Line 263: Line 284:




=== <span style="color: purple;">''/Logger/Channels/0/Settings/Byte limit''</span> ===
=== <span style="color: purple;">''Byte limit''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' DOUBLE
* '''Type:''' DOUBLE
Line 270: Line 291:


This field can be used to stop a run when set to a non-zero value. The statistics value Bytes written is checked against the limit Byte limit . When this conditions is reached, the run is stopped automatically by the logger. The next run can then be started automatically if desired (see Auto restart ).
The  value of this ODB key <span style="color: purple; font-style:italic;">Byte limit</span>  in the [[#Settings subtree|Settings subtree]]  can be used to stop a run when set non-zero. The statistics value [[#Bytes written|Bytes written]] is checked against the <span style="color: purple; font-style:italic;">Byte limit</span> value. When this conditions is reached, the run is stopped automatically by the data logger [[Mlogger|mlogger]]. The next run can then be started automatically if desired (see [[Keys in the ODB /Logger tree #/Logger/Auto restart|Auto restart]]). ).


<br>
<br>
Line 276: Line 297:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Subrun Byte limit''</span> ===
=== <span style="color: purple;">''Subrun Byte limit''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' DOUBLE
* '''Type:''' DOUBLE
Line 283: Line 304:




This field can be used to stop a [[subruns|subrun]] when set to a non-zero value. When this condition is reached, the run is stopped automatically by the MIDAS logger ([[mlogger]]). The next run can then be started automatically if desired (see [[Keys in the ODB /Logger tree #/Logger/Auto restart |/Logger/Auto restart]]).
The value of the ODB key <span style="color: purple; font-style:italic;">Subrun Byte limit</span>  in the [[#Settings subtree|Settings subtree]] can be used to stop a [[subruns|subrun]] when set non-zero. When this condition is reached, the run is stopped automatically by the data logger ([[mlogger]]). The next run can then be started automatically if desired (see [[Keys in the ODB /Logger tree #/Logger/Auto restart |Auto restart]]).


<br>
<br>
-----
-----
<br>
<br>
=== <span style="color: purple;">''/Logger/Channels/0/Settings/Tape capacity''</span> ===
=== <span style="color: purple;">''Tape capacity''</span> (obsolete) ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' DOUBLE
* '''Type:''' DOUBLE
Line 294: Line 315:
</div>  
</div>  


This field can be used to stop a run when set to a non-zero value. The statistics value Bytes written total in the is checked against the limit Tape capacity. When this conditions is reached, the run is stopped automatically by the logger.
The value of the ODB key <span style="color: purple; font-style:italic;">Tape capacity</span>  in the [[#Settings subtree|Settings subtree]]  can be used to stop a run when set to non-zero. The value in the key [[#Bytes written total|Bytes written total]] is checked against the limit set by <span style="color: purple; font-style:italic;">Tape capacity</span>. When this condition is reached, the run is stopped automatically by the data logger.
 
<div style="background-color:mistyrose;">Warning: writing directly to tape is not maintained</div>
<br>
<br>
-----
-----
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Settings/Subdir format''</span> ===
=== <span style="color: purple;">''Subdir format''</span> ===
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Type:''' STRING
Line 307: Line 328:




This field optionally specifies a subdirectory, which is appended to the directory string in the key Filename.
This ODB key  in the [[#Settings subtree|Settings subtree]] optionally specifies a subdirectory, which is appended to the directory string in the key [[Keys in the ODB /Logger/Channels tree #Filename | Filename]].


It is used to place the data log file into a date-stamped subdirectory. This is useful where a large numbers of datafiles are produced, or experiments are active for a long time period, and it is desirable to store the data in sub-directories according to their date. This can be done automatically using the Subdir format field.
It is used to place the data log file into a date-stamped subdirectory. This is useful where a large numbers of data files are produced, or experiments are active for a long time, and it is desirable to store the data in sub-directories according to their date. This can be done automatically using the <span style="color: purple; font-style:italic;">Subdir format</span> key.


If the  <span style="color: purple;">''/Logger/Channels/0/Settings/Subdir format''</span> field is not empty, the data log file will be placed in a subdirectory. The name of this subdirectory is composed by the given <span style="color: purple;">''/Logger/Channels/0/Settings/Subdir format''</span>  string. Its format follows the definition of the system call strftime() . (See man strftime). Ordinary characters placed in the format string are copied to the output character array "s" without conversion. Conversion specifiers are introduced by a '%' character, and are replaced in "s" as follows for the most common ones:
If the  <span style="color: purple;">''Subdir format''</span> key is not empty, the data log file will be placed in a subdirectory. The name of this subdirectory is composed by the given <span style="color: purple;">''Subdir format''</span>  string. Its format follows the definition of the system call strftime() . (See man strftime). Ordinary characters placed in the format string are copied to the output character array "s" without conversion. Conversion specifiers are introduced by a '%' character, and are replaced in "s" as follows for the most common ones:


*    %Y : Year (ex: 2002)
*    %Y : Year (ex: 2002)
Line 325: Line 346:




=== <span style="color: purple;">''/Logger/Channels/0/Settings/Current filename''</span> ===
=== <span style="color: purple;">''Current filename''</span> ===


<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
Line 332: Line 353:
</div>  
</div>  


This key will reflect the full path of the current saved data file.
This ODB key in the [[#Settings subtree|Settings subtree]]  will reflect the full path of the current saved data file.
 
<br>
-----
<br>
<br>
 
=== <span style="color: purple;">''Data checksum''</span> ===
 
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:''' CRC32C
</div>
 
This ODB key in the [[#Settings subtree|Settings subtree]] can be set to
one of: NONE CRC32C SHA256 SHA512 ZLIB.
 
See Note on defaults under [[#Compress]] below.
<br>
-----
<br>
<br>
 
=== <span style="color: purple;">''File checksum''</span> ===
 
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:''' CRC32C
</div>
 
This ODB key in the [[#Settings subtree|Settings subtree]] can be set to one of: NONE CRC32C SHA256 SHA512 ZLIB.


See Note on defaults under [[#Compress]] below
<br>
<br>
-----
-----
Line 339: Line 391:
<br>
<br>


== <span style="color: purple;">''/Logger/Channels/0/Statistics''</span> subtree ==
=== <span style="color: purple;">''Compress''</span> ===
The keys in the ODB <span style="color: purple;">''/Logger/Channels/<logging-channel-number>/Settings''</span> subtree are shown below
 
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:''' lz4
</div>
 
This ODB key in the [[#Settings subtree|Settings subtree]] can be set to one of :  none gzip lz4 bzip2 pbzip2.
 
;Note
The defaults of "free" compression (LZ4) and output file data integrity protected by checksums (CRC32C)
are "free" in the sense that turning
them off would not noticeably improve the system performance. (Some users may want to enable better compression,
such as BZIP2 or PBZIP2 and better checksums, such as SHA256 or SHA512 - both choices that have significant CPU-use cost).
<br>
-----
<br>
<br>
 
=== <span style="color: purple;">''Output''</span> ===
 
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
* '''Type:''' STRING
* '''Default:''' FILE
</div>
 
This ODB key in the [[#Settings subtree|Settings subtree]] can be set to one of: NULL FILE FTP ROOT PIPE.
 
 
 
 
== <span style="color: purple;">''Statistics''</span> subtree ==
The keys in the ODB <span style="color: purple;">''/Logger/Channels/<logging-channel-number>/Statistics''</span> subtree are shown below
for the default logging channel 0.
for the default logging channel 0.


For a single logging channel 0, one subtree
For a single logging channel 0, one ODB subtree
* <span style="color: purple;">''/Logger/Channels/0/Settings''</span>  
* <span style="color: purple;">''/Logger/Channels/0/Statistics''</span>  
will be present.
will be present.




If [[#Multiple Logging Channels | multiple logging channels]] are defined, each
If [[#Multiple Logging Channels | multiple logging channels]] are defined, each
will contain its own ODB <span style="color: purple;">''Settings''</span>  subtree,  
will contain its own ODB <span style="color: purple;">''Statistics''</span>  subtree,  


i.e.  
i.e.  
* <span style="color: purple;">''/Logger/Channels/0/Settings''</span>  
* <span style="color: purple;">''/Logger/Channels/0/Statistics''</span>  
* <span style="color: purple;">''/Logger/Channels/1/Settings''</span>  
* <span style="color: purple;">''/Logger/Channels/1/Statistics''</span>  


<br>
<br>
Line 359: Line 442:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Statistics/Events written''</span> ===
=== <span style="color: purple;">''Events written''</span> ===


<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
Line 366: Line 449:
</div>  
</div>  


Number of events written. Cleared at begin-of-run. <span style="color: purple;">''Events written''</span>  statistics are used to determine when to stop a run automatically by checking against the [[#/Logger/Channels/0/Settings/Event limit|Event limit key]].
This ODB key  in the [[#Statistics subtree|Statistics subtree]]  contains the number of events written. Cleared at begin-of-run. <span style="color: purple;">''Events written''</span>  statistics are used to determine when to stop a run automatically by checking against the [[#/Logger/Channels/0/Settings/Event limit|Event limit key]].


<br>
<br>
Line 372: Line 455:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Statistics/Bytes written''</span> ===
=== <span style="color: purple;">''Bytes written''</span> ===


<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
Line 379: Line 462:
</div>  
</div>  


Number of bytes written. Cleared at begin-of-run. <span style="color: purple;">''Bytes written''</span> statistics are used to determine when to stop a run automatically by checking against the [[#/Logger/Channels/0/Settings/Byte limit|Byte limit]] key.
 
This ODB key  in the [[#Statistics subtree|Statistics subtree]] contains the number of bytes written. Cleared at begin-of-run. <span style="color: purple;">''Bytes written''</span> statistics are used to determine when to stop a run automatically by checking against the [[#Byte limit|Byte limit]] key.




Line 386: Line 470:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Statistics/Bytes written uncompressed''</span> ===
=== <span style="color: purple;">''Bytes written uncompressed''</span> ===


<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
Line 393: Line 477:
</div>  
</div>  


If [[#/Logger/Channels/0/Settings/Compression |Compression]] level is 0 (i.e. data not compressed), the values of  
If [[#Compression |Compression level]] is 0 (i.e. data not compressed), the values of the ODB keys  in the [[#Statistics subtree|Statistics subtree]]
 
<span style="color: purple;">''Bytes written uncompressed''</span> and [[#/Logger/Channels/0/Settings/Bytes written|Bytes written]] will be identical.  
<span style="color: purple;">''Bytes written uncompressed''</span> and [[#/Logger/Channels/0/Settings/Bytes written|Bytes written]] will be identical.  


Line 402: Line 485:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Statistics/Bytes written total''</span> ===
=== <span style="color: purple;">''Bytes written total''</span> ===


<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
Line 410: Line 493:




Total number of bytes written. Cleared only when a tape is rewound with the [[odbedit]] command ''rewind''. The <span style="color: purple;">''Bytes written total''</span> entry can therefore be used as an indicator of when a tape is full by checking against the limit [[#/Logger/Channels/0/Settings/Tape Capacity|Tape Capacity]].
This ODB key  in the [[#Statistics subtree|Statistics subtree]] contains the total number of bytes written. Cleared only when a tape is rewound with the [[odbedit]] command ''rewind''. The <span style="color: purple;">''Bytes written total''</span> entry can therefore be used as an indicator of when a tape is full by checking against the limit [[#/Logger/Channels/0/Settings/Tape Capacity|Tape Capacity]].
 
<div style="background-color:mistyrose;">Warning: writing directly to tape is not maintained</div>


<br>
<br>
Line 417: Line 500:
<br>
<br>


=== <span style="color: purple;">''/Logger/Channels/0/Statistics/Files written''</span> ===
=== <span style="color: purple;">''Files written''</span> ===


<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
<div style="column-count:3;-moz-column-count:3;-webkit-column-count:3">
Line 424: Line 507:
</div>  
</div>  


Number of files written. Cleared only when a tape is rewound with the [[odbedit]] command ''rewind''. <span style="color: purple;">''Files written''</span> value can be used off-line to determine how many files on a tape have to be skipped in order to reach a specific run.
This ODB key in the [[#Statistics subtree|Statistics subtree]]  contains the number of files written. Cleared only when a tape is rewound with the [[odbedit]] command ''rewind''. <span style="color: purple;">''Files written''</span> value can be used off-line to determine how many files on a tape have to be skipped in order to reach a specific run.
<div style="background-color:mistyrose;">Warning: writing directly to tape is not maintained</div>
 
 
[[Category:Logger Tree]]

Latest revision as of 16:09, 22 May 2019

Links


/Logger/Channels subtree

The ODB /Logger/Channels subtree is a subdirectory of the ODB /Logger tree, created when the MIDAS logger utility mlogger is started for the first time. The keys will be filled with default values which users may change as required to customize their experiments.

By default, only one logging channel with the value 0 if defined (i.e. Logging Channel 0). Multiple logging channels may be defined by the user.




Multiple Logging Channels

By default, only one logging channel is created (Logging Channel 0). To create multiple logging channels, an existing Channel subtree can be copied.

e.g. to create a second logging channel (Channel 1) using odbedit

[local]cd /Logger
[loca]]logger>cd channels
[local]Channels>ls 
0
[local]Channels>copy 0 1
[local]Channels>ls 
0
1

If more than one logging channel is defined, there will be a multiple subtrees

  • /Logger/Channels/0/,
  • /Logger/Channels/1/,
  • ...

present, each with a Settings and Statistics subtree.

Thus the parameters for each logging channel can be set individually.

To send different events to different logging channels, they may be selected by Event ID key or Trigger mask key.




Settings subtree

The keys in the ODB /Logger/Channels/<logging-channel-number>/Settings subtree are shown below for the default logging channel 0.

For a single logging channel 0, one subtree

  • /Logger/Channels/0/Settings

will be present.


If multiple logging channels are defined, each will contain its own ODB Settings subtree,

i.e.

  • /Logger/Channels/0/Settings
  • /Logger/Channels/1/Settings




Active

  • Type: BOOL
  • Default: "y"

If this ODB key in the Settings subtree is set to "y", this logging channel is active. Data are only logged from channels that are active. Set to "n" to disable data logging from this channel.

Note that for data to be logged from any active logging channel, the /Logger/Write data key must also be set to "y".




Type

  • Type: STRING
  • Default: "Disk"

This ODB key in the Settings subtree specifies the type of media on which the logging should take place. It can be "Disk", "Tape" or "FTP" (see Filename key).




Filename

  • Type: STRING
  • Default: "run%05d.mid"

This ODB key in the Settings subtree is set up as follows:

For disk logging
ODB key Filename specifies the name of a file to contain the data. Default value is "run%05d.mid" where "%05d" is replaced by the current run number in the same way as described for the ODB key /Logger/Dump File. If subruns are enabled, the filename may be of the form "run%05d_%02d.mid". See subruns for details.
The key Filename may contain a directory string in addition to a filename, so that different logging channels may have their data logged into specific directories. If the key does NOT also contain a directory string, the directory given in the key /Logger/Data dir key will be used. The contents of the key Filename must match that of the Format key.
For tape logging
key Filename specifies a tape device i.e.
  • /dev/nrmt0 or /dev/nst0 under UNIX or
  • \\.\tape0 under Windows NT.
WARNING: tape logging is no longer maintained
FTP mode
the data is written directly to a remote computer via FTP. Key Filename specifies the access information for the FTP server. It has the following format:
 <host name>, <port number>, <user name>, <password>, <directory>, <file name>
for example
 myhost.my.domain,21,john,password,/usr/users/data,run%05d.mid

The normal FTP port number is 21, or 1021 for a Unitree Archive like the one used at the Paul Scherrer Institute.

By using the FTP mode, a back-end computer can write directly to the archive.




Format

  • Type: STRING
  • Default: "MIDAS"

This ODB key in the Settings subtree specifies the format to be used for writing the data to the logging channel. It can take one of four values: MIDAS, ROOT, ASCII and DUMP.

For disk logging, the extension for the Filename key above must match one of the following:

  • .mid for MIDAS (or .mid.gz - see Compression key)
  • .root for ROOT
  • .asc for ASCII
  • .txt for DUMP

For ROOT format the data logger will automatically convert the format specified in the frontend equipment list to ROOT format (see format for details). The ASCII format converts events into readable text format which can be easily analyzed by programs that have problems reading binary data. While the ASCII format tries to minimize the file size by printing one event per line, the DUMP format gives a very detailed ASCII representation of the event including bank information, serial numbers etc., so it should be used mainly for diagnostics.




Compression

  • Type: INT
  • Default: 0


This ODB key in the Settings subtree defines the Compression Level of the data. If set to 0, compression is turned off. If set > 0, the data files are compressed before being written. In this case,

  • the file extension ".gz" must be appended to the Filename key (see also Format key)
  • and ZLIB support must be compiled into the MIDAS package (see NEED_ZLIB ).

This feature applies only to files saved onto disk in MIDAS format. ROOT format is already compressed.




ODB Dump

  • Type: BOOL
  • Default: "y"


If this ODB key in the Settings subtree is set to "y", a complete dump of the ODB is performed to this logging channel before and after every run. The format of the ODB dump is set by the key #ODB dump format. The ODB content is reflects the status at begin-of-run event and at end-of-run event. These special events have an ID of EVENTID_BOR (0x8000) and EVENTID_EOR (0x8001) (defined in midas.h) and a serial number equal to the current run number, and are not visible using the mdump utility. An analyzer in the off-line analysis stage can use these special events to restore the ODB to its online state.

The dump of the ODB into the event stream for this logger channel is independent of the /Logger/ODB Dump key, which is used to dump the ODB to a file on disk in the specified format.




ODB dump format

  • Type: STRING
  • Default: "json"

This ODB key in the Settings subtree defines the format of the ODB Dump file, providing the ODB key #ODB Dump is set to "y". The default is now "json" (since March 2019) and the filename of the odb dump file saved at end-of-run is is "last.json".

Users who wish to use "xml" should set this key to "xml" and Keys in the ODB /Logger tree#ODB Last Dump File to "last.xml".




Log messages

  • Type: DWORD
  • Default: 0

This ODB key in the Settings subtree is a bit-field for logging system messages as message events along with the event data from the experiment. If a bit in this field is set, the respective system message is written to the logging channel as a message event with an ID of EVENTID_MESSAGE (0x8002) (defined in midas.h). The bits are 1 for error, 2 for info, 4 for debug, 8 for user, 16 for log, 32 for talk, 64 for call messages and 255 to log all messages. For an explanation of these messages refer to Buffer Manager , Event ID key and Trigger mask key.






Buffer

  • Type: STRING
  • Default: "SYSTEM"


This ODB key in the Settings subtree specifies the buffer where the logger will find the event. It is usually set to "SYSTEM" and is specified in the equipment list for this event.




Event ID

  • Type: INT
  • Default: -1


This ODB key in the Settings subtree contains the ID of the event(s) to be logged (see EventID). It is specified in the Equipment List for this event.

To log events with any ID, Event ID is set to -1 (EVENTID_ALL). Event ID key and/or Trigger mask key may be used to select events to be sent to different logging channels, or as an Event Filtering system.




Trigger mask

  • Type: INT
  • Default: -1

This ODB key in the Settings subtree specifies which events to log based on the Trigger Mask of the event. It is specified in the Equipment List of this event.

For a logging channel to receive all events, -1 is used for both the Event ID key and the Trigger mask key .

Either the Trigger Mask or Event ID key may be used to select events to be sent to particular logging channels, or for Event Filtering.




Event limit

  • Type: DOUBLE
  • Default: 0


The value of this ODB key Event limit in the Settings subtree can be used to stop a run when set to a non-zero value. The statistics value Events written is checked against the Event limit value. When this condition is reached, the run is stopped automatically by the data logger mlogger. The next run can then be started automatically if desired (see Auto restart).





Byte limit

  • Type: DOUBLE
  • Default: 0


The value of this ODB key Byte limit in the Settings subtree can be used to stop a run when set non-zero. The statistics value Bytes written is checked against the Byte limit value. When this conditions is reached, the run is stopped automatically by the data logger mlogger. The next run can then be started automatically if desired (see Auto restart). ).




Subrun Byte limit

  • Type: DOUBLE
  • Default: 0


The value of the ODB key Subrun Byte limit in the Settings subtree can be used to stop a subrun when set non-zero. When this condition is reached, the run is stopped automatically by the data logger (mlogger). The next run can then be started automatically if desired (see Auto restart).




Tape capacity (obsolete)

  • Type: DOUBLE
  • Default: 0

The value of the ODB key Tape capacity in the Settings subtree can be used to stop a run when set to non-zero. The value in the key Bytes written total is checked against the limit set by Tape capacity. When this condition is reached, the run is stopped automatically by the data logger.

Warning: writing directly to tape is not maintained




Subdir format

  • Type: STRING
  • Default:


This ODB key in the Settings subtree optionally specifies a subdirectory, which is appended to the directory string in the key Filename.

It is used to place the data log file into a date-stamped subdirectory. This is useful where a large numbers of data files are produced, or experiments are active for a long time, and it is desirable to store the data in sub-directories according to their date. This can be done automatically using the Subdir format key.

If the Subdir format key is not empty, the data log file will be placed in a subdirectory. The name of this subdirectory is composed by the given Subdir format string. Its format follows the definition of the system call strftime() . (See man strftime). Ordinary characters placed in the format string are copied to the output character array "s" without conversion. Conversion specifiers are introduced by a '%' character, and are replaced in "s" as follows for the most common ones:

  • %Y : Year (ex: 2002)
  • %y : Year (range:00..99)
  • %m : Month (range: 01..12)
  • %d : Day (range: 00..31)

The other characters are: %a, %A, %b, %B, %c, %C, %d, %D, %e, %E, %G, %g, %h, %H, %I, %j, %k, %l, %m, %M, %n, %O, %p, %P, %r, %R, %s, %S, %t, %T, %u, %U, %V, %w, %W, %x, %X, %y, %Y, %z, %Z, %+. (See man strftime for explanation).





Current filename

  • Type: STRING
  • Default:

This ODB key in the Settings subtree will reflect the full path of the current saved data file.





Data checksum

  • Type: STRING
  • Default: CRC32C

This ODB key in the Settings subtree can be set to one of: NONE CRC32C SHA256 SHA512 ZLIB.

See Note on defaults under #Compress below.




File checksum

  • Type: STRING
  • Default: CRC32C

This ODB key in the Settings subtree can be set to one of: NONE CRC32C SHA256 SHA512 ZLIB.

See Note on defaults under #Compress below




Compress

  • Type: STRING
  • Default: lz4

This ODB key in the Settings subtree can be set to one of : none gzip lz4 bzip2 pbzip2.

Note

The defaults of "free" compression (LZ4) and output file data integrity protected by checksums (CRC32C) are "free" in the sense that turning them off would not noticeably improve the system performance. (Some users may want to enable better compression, such as BZIP2 or PBZIP2 and better checksums, such as SHA256 or SHA512 - both choices that have significant CPU-use cost).




Output

  • Type: STRING
  • Default: FILE

This ODB key in the Settings subtree can be set to one of: NULL FILE FTP ROOT PIPE.



Statistics subtree

The keys in the ODB /Logger/Channels/<logging-channel-number>/Statistics subtree are shown below for the default logging channel 0.

For a single logging channel 0, one ODB subtree

  • /Logger/Channels/0/Statistics

will be present.


If multiple logging channels are defined, each will contain its own ODB Statistics subtree,

i.e.

  • /Logger/Channels/0/Statistics
  • /Logger/Channels/1/Statistics




Events written

  • Type: DOUBLE
  • Default: 0

This ODB key in the Statistics subtree contains the number of events written. Cleared at begin-of-run. Events written statistics are used to determine when to stop a run automatically by checking against the Event limit key.




Bytes written

  • Type: DOUBLE
  • Default: 0


This ODB key in the Statistics subtree contains the number of bytes written. Cleared at begin-of-run. Bytes written statistics are used to determine when to stop a run automatically by checking against the Byte limit key.





Bytes written uncompressed

  • Type: DOUBLE
  • Default: 0

If Compression level is 0 (i.e. data not compressed), the values of the ODB keys in the Statistics subtree Bytes written uncompressed and Bytes written will be identical.





Bytes written total

  • Type: DOUBLE
  • Default: 0


This ODB key in the Statistics subtree contains the total number of bytes written. Cleared only when a tape is rewound with the odbedit command rewind. The Bytes written total entry can therefore be used as an indicator of when a tape is full by checking against the limit Tape Capacity.

Warning: writing directly to tape is not maintained




Files written

  • Type: INT
  • Default: 0

This ODB key in the Statistics subtree contains the number of files written. Cleared only when a tape is rewound with the odbedit command rewind. Files written value can be used off-line to determine how many files on a tape have to be skipped in order to reach a specific run.

Warning: writing directly to tape is not maintained