Experimental Physics and
Industrial Control System

[email protected] (Jeff Hill) · Thu, 18 Nov 1999 09:55:48 -0700

Chris,

My only comment is that similar functionality is planned in CA V4
(ala my paper/poster from ICALEPCS 99 and my talk at the last
two collaboration meetings), and so there is some potential for 
redundant code in the IOC. Perhaps there is some way that we can 
combine our efforts?

Jeff

> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Sent: Wednesday, November 17, 1999 5:43 PM
> Cc: [email protected]
> Subject: Re: Channel Archiving
> 
> 
> Hi all, 
> 
>   [the following is fairly long and perhaps tedious,
>    so here's the executive summary]
> 
>   I'm working on a data logger which will run on an
>   ioc, compress channel data in real-time, and stream 
>   it over the network to a spooler.  I hope that the 
>   resulting system can be generally useful to other 
>   sites, and so need some small amount of information
>   and/or comments.
> 
>   [end executive summary, but there's some more good 
>    stuff at the bottom]
> 
>   (1) What is the fastest rate at which people want to
>       log data from an ioc?
> 
>   (2) How much of this data is there?
> 
>   My preliminary investigation of how data behaves on
> the ioc's here at JLAB indicates that, if we consider
> each byte which comprises a signal's value as an
> independent quantity, the probability that any particular
> one of these will change in a ten minute period ranges
> between 0.005 and 0.05 (this is sampling at 1 hz).  
> Futhermore, when a byte does change, it does so in a 
> remarkably structured manner  (see
> http://www.sura.org/~larrieu/Info.html).  
> From an information theory standpoint, this means that 
> 
>   (1) the observation that a signal has not changed 
>       carries very little information and can therefore 
>       be conveyed with minimal overhead.
> 
>   (2) when a byte does change, it will probably change
>       to one of the predictable values, so this too can 
>       be encoded using very low bit rates.
> 
>   (3) when a byte changes to an unlikely value, then
>       we need more bits to indicate such, though this
>       happens relatively infrequently.
> 
>   Of course, when you know that a certain byte belongs
> to a certain signal, then you know more about its 
> behavior over time.  By monitoring this behavior we
> can build a fairly accurate statistical model of it
> so that we can further reduce the amount of information
> required to indicate a change in value.
> 
>   The whole point here is that I believe we can achieve
> dramatic results by compressing the channel data on the
> ioc, then sending the result over the wire to a simple
> spooling process.
> 
>   My intent is to:
> 
>   (1) implement this logging system
>   (2) implement a client library for accessing the logged
>       data, using Kay's C++ API.
>   (3) build a new data extraction tool using (2)
> 
>   The benefits should be:
> 
>   (1) ability to store more information with less data
>   (2) significantly reduced network traffic
>   (3) circumvent CA size limit
>   (4) compatible high-level interface, facilitating
>       sharing of tools between participating labs.
> 
>   Trade-offs:
> 
>   (1) increased memory requirements on ioc
>   (2) possibly significant CPU time on ioc
>   (3) increased access time to archived data (decompress)
>   (4) annoyance: another archiver?!
>  
> Chris
> -- 
> Christopher A. Larrieu
> Jefferson Laboratory
> (757) 269-5097
> 

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System