Experimental Physics and
Industrial Control System

For clarity and completeness, I probably should add that this software is not in the
regime suggested in Jeff's reply (intermediate events discarded; producer outpaces
consumer).  That's not possible in this case, because the events are what drive the
software from one state to the next; their number and sequence is known, and all of
them appear in the diagnostic output.

Hi All,

I read Tim's initial post with interest because I am sure that I was told at an early EPICS training course (probably in 1992 or 93) that monitors posted to a given client within a given lockset were guaranteed to be received in order (and, conversely, ca_puts from a client were delivered in order). I have since used this for a lot of systems that need to collect a coherent block of information. I have not known it to fail, but I haven't looked for the 1 in a million problem, just the 1 in 100 or so. Now this old wives tale is clearly being challenged.

I find Jeff's response ambiguous, and maybe that's the problem. Jeff says there is a per client event queue. Tim seems to imply it is per-subscription. Ralf seems to imply the truth is somewhere in between, and since it is more specific it has the ring of truth about it.

Was my memory ever true? I have a suspicion that by the tone of his message Tim may have been labouring under the same misapprehension. It may have changed, for example, when we lifted the fixed (read 16k) CA limit. I would agree with Tim in that in my experience it is true most of the time - especially on vxWorks, and since I have mostly used it to get synchronised hard real-time data from a server to a client, vxWorks is where I have used it.

I am not sure what to do about it now, but I definitely have a frequent requirement to get a coherent set of data from a server. Up till now, I have monitored everything and then taken a snapshot whenever I got a monitor from the last item in a processing chain (which I have ensured always updates whenever the chain processes). This has seemed to work with VxWorks servers. I would like to still be able to achieve the same effect. How else can I do this?

Cheers,

Nick Rees
Principal Software Engineer           Phone: +44 (0)1235-778430
Diamond Light Source                  Fax:   +44 (0)1235-446713

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jeff Hill
Sent: 01 November 2010 21:42
To: 'Tim Mooney'; 'EPICS tech-talk'
Subject: RE: monitors received out of order

Hi Tim,

The Channel Access publish and subscribe system is a classic
consumer/producer situation. The producer is of course record
processing and the consumer is the Channel Access client side
application. In basic queuing theory, it is known that no
matter how big the finite buffer capacity between the producer
and the consumer, all of it will be consumed if the production
rate is even a marginal amount faster than the consumption rate.
Furthermore, to prevent one client from disrupting subscription
updates sent to another client, each client attaching to an IOC
is serviced out of a per-client dedicated event queue connecting
the higher priority database processing threads with the lower
priority per-client server threads. So to avoid consuming all
available memory this event queue is finite length, and if full
capacity is reached the last event on the queue for a particular
subscription is replaced; this has the net impact of causing an
intermediate subscription update to be discarded.

So one can easily see, given the above design with intermediate
events being discarded when the producer out-paces the consumer,
that events will sometimes be delivered out of order when comparing
different channel subscriptions, but that events will never be
delivered out of order within a single channel subscription.

HTH

Jeff
______________________________________________________
Jeffrey O. Hill           Email        [email protected]
LANL MS H820              Voice        505 665 1831
Los Alamos NM 87545 USA   FAX          505 665 5107

Message content: TSPA

With sufficient thrust, pigs fly just fine. However, this is
not necessarily a good idea. It is hard to be sure where they
are going to land, and it could be dangerous sitting under them
as they fly overhead. -- RFC 1925

-----Original Message-----
From: [email protected] [mailto:[email protected]]
On Behalf Of Tim Mooney
Sent: Monday, November 01, 2010 3:10 PM
To: EPICS tech-talk
Subject: monitors received out of order

Dear folks,

I have two records, and a separate task monitoring a field that both
records
post, all in the same IOC.  Most of the time, my task receives monitors in
the
order in which they were posted.  But sometimes, the task receives a
monitor
from record B before it receives a previously posted monitor from record
A.
(I know for sure which record is posting first, because record A posts
before
causing record B to process.  Also, I've modified the record to set its
time
stamp immediately before posting, and I get another time stamp on entry to
the monitor routine.  The time ordering of those stamps does not agree.)
I've seen this on solaris and Linux, but not on vxWorks.

I have code that misbehaves when this happens, so I started digging around
and have convinced myself that I should not be relying on the time
ordering of
monitors received from different records (even when those records are
running
in the same task).  I now think I can rely on posts from a single record
arriving in
time order, but not posts from different records.  I think this because
events from
different record go into different queues, and there doesn't seem to be
any code
in the vicinity that seems worried about time ordering across event
queues.

Am I right about this?

--
Tim Mooney ([email protected]) (630)252-5417
Software Services Group, Advanced Photon Source, Argonne National Lab.

--
Tim Mooney ([email protected]) (630)252-5417
Software Services Group, Advanced Photon Source, Argonne National Lab.

References:

monitors received out of order Tim Mooney

RE: monitors received out of order Jeff Hill

RE: monitors received out of order nick.rees

Navigate by Date:

Prev: RE: monitors received out of order Jeff Hill

Next: Re: monitors received out of order Tim Mooney

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <2010>  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: RE: monitors received out of order Jeff Hill

Next: Re: monitors received out of order Ralph Lange

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <2010>  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024