Experimental Physics and
Industrial Control System

Michael Abbott <[email protected]> · Tue, 27 Feb 2007 07:25:00 +0000 (GMT)

On Mon, 26 Feb 2007, Andrew Johnson wrote:

You can use the epicsAtExit() facility to get callbacks when shutdown is
occurring, but the current state of the other threads when this happens is
not well-defined, meaning your I/O thread may be active at the time and
there's nothing we can do about that.

Ok ... and alas. By "your I/O thread" I suspect you actually mean EPICS
threads? As I say, I have full control over my own threads! If there was
a call I could hook in to say "from this point there will be no further
EPICS initiated activity (calls from EPICS threads)" it would be perfect.
If not then I'll just make sure to batten down the hatches so I don't
crash during shutdown (grr!) and rely on the OS to tidy up the rest.
I hope you've heard of the GNU Screens program

Yes, screen is excellent ... but oddly enough, not terribly useful in my
current application. The earlier versions of my software (yes, the Libera
EPICS driver) ran in a screen, but I never found it useful. I find I have
two modes of operation: single machine development in the lab, when I run
the IOC interactively; and deployed onto the accelerator, when I run 205
copies as daemons writing to a log file.
The problems I encounter are either easy logic problems, which can be
sorted out during interactive development on a single machine, or hair
tairing intermittent problems which happen on one machine in a hundred
every few hours ... I now have 32 core dumps to puzzle over :(

Here's a very odd problem that's been bothering me (sorry, bit of a
digression here): I have processing chains of the form
 bi -> fanout -> (r1, ..., rn, bo)
where bi is a SCAN="I/O Intr" record that I trigger when records r1 to rn
need to update and bo is a record that gets processed when records r1 to
rn have finished. In the IOC itself I run the following loop (obviously
with the appropriate setup so that (2) can come first!):
(1)	wait for external event
(2)	wait for bo record above to process
(3)	update underlying data for r1 to rn
(4)	signal bi record (scanIoRequest)
I have half a dozen or so instances of this pattern which works very
nicely 99.9999...% of the time ... but every now and then something very
odd happens: step (2) above times out on one instance ... and another
instance gets an EXTRA signal. It's as if the bo record from one chain
was processed during the processing of a completely different chain! Or
perhaps my semaphores are wonky? Of course (2) and (4) are linked by a
pthread semaphore.
As this effect is really very rare, I can only think that it's a
synchronisation problem. My suspicion current falls in one of two places:
the bowels of EPICS record processing, or something fundamental in the
threading library I'm using (yes, I'm still using gcc 3.3.3 with an old
glibc on the ARM).

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System