Experimental Physics and
Industrial Control System

"Christopher A. Larrieu" <[email protected]> · Thu, 18 Nov 1999 14:44:51 -0500

Jeff Hill wrote:
> 
> Chris,
> 
> My only comment is that similar functionality is planned in CA V4
> (ala my paper/poster from ICALEPCS 99 and my talk at the last
> two collaboration meetings), and so there is some potential for
> redundant code in the IOC. Perhaps there is some way that we can
> combine our efforts?

  Certainly!  I hadn't realized that you were actually working
on this already, and thought that it was a "gee it would be nice
if someone would do this" sort of deal.  As I am just starting,
perhaps we should discuss our ideas and try to synthesize some-
thing from them.  Because this work is precipitated by course- 
related research, my immediate goals are a precise understanding
of the problem and a formulation of the solution, with the
implementation being secondary; once I've arrived at this under-
standing, though, the implementation should follow in a fairly
straightforward fashion (really, I have no naive expectation that 
it will be *easy*)...

Chris

> 
> Jeff
> 
> > -----Original Message-----
> > From: [email protected] [mailto:[email protected]]
> > Sent: Wednesday, November 17, 1999 5:43 PM
> > Cc: [email protected]
> > Subject: Re: Channel Archiving
> >
> >
> > Hi all,
> >
> >   [the following is fairly long and perhaps tedious,
> >    so here's the executive summary]
> >
> >   I'm working on a data logger which will run on an
> >   ioc, compress channel data in real-time, and stream
> >   it over the network to a spooler.  I hope that the
> >   resulting system can be generally useful to other
> >   sites, and so need some small amount of information
> >   and/or comments.
> >
> >   [end executive summary, but there's some more good
> >    stuff at the bottom]
> >
> >   (1) What is the fastest rate at which people want to
> >       log data from an ioc?
> >
> >   (2) How much of this data is there?
> >
> >   My preliminary investigation of how data behaves on
> > the ioc's here at JLAB indicates that, if we consider
> > each byte which comprises a signal's value as an
> > independent quantity, the probability that any particular
> > one of these will change in a ten minute period ranges
> > between 0.005 and 0.05 (this is sampling at 1 hz).
> > Futhermore, when a byte does change, it does so in a
> > remarkably structured manner  (see
> > http://www.sura.org/~larrieu/Info.html).
> > From an information theory standpoint, this means that
> >
> >   (1) the observation that a signal has not changed
> >       carries very little information and can therefore
> >       be conveyed with minimal overhead.
> >
> >   (2) when a byte does change, it will probably change
> >       to one of the predictable values, so this too can
> >       be encoded using very low bit rates.
> >
> >   (3) when a byte changes to an unlikely value, then
> >       we need more bits to indicate such, though this
> >       happens relatively infrequently.
> >
> >   Of course, when you know that a certain byte belongs
> > to a certain signal, then you know more about its
> > behavior over time.  By monitoring this behavior we
> > can build a fairly accurate statistical model of it
> > so that we can further reduce the amount of information
> > required to indicate a change in value.
> >
> >   The whole point here is that I believe we can achieve
> > dramatic results by compressing the channel data on the
> > ioc, then sending the result over the wire to a simple
> > spooling process.
> >
> >   My intent is to:
> >
> >   (1) implement this logging system
> >   (2) implement a client library for accessing the logged
> >       data, using Kay's C++ API.
> >   (3) build a new data extraction tool using (2)
> >
> >   The benefits should be:
> >
> >   (1) ability to store more information with less data
> >   (2) significantly reduced network traffic
> >   (3) circumvent CA size limit
> >   (4) compatible high-level interface, facilitating
> >       sharing of tools between participating labs.
> >
> >   Trade-offs:
> >
> >   (1) increased memory requirements on ioc
> >   (2) possibly significant CPU time on ioc
> >   (3) increased access time to archived data (decompress)
> >   (4) annoyance: another archiver?!
> >
> > Chris
> > --
> > Christopher A. Larrieu
> > Jefferson Laboratory
> > (757) 269-5097
> >

-- 
Christopher A. Larrieu
Jefferson Laboratory
(757) 269-5097

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System