Experimental Physics and
Industrial Control System

Richard Dickson <[email protected]> · Thu, 16 Mar 2000 08:46:29 -0500

Chip,
    A problem exists with elevating the CA TCP task priority that has caused
us to not run in that configuration for some time now.  It has happened
that an EPICS processing task (such as a scan or sequence task) becomes
stuck in an infinite loop polling something that never responds.  You can
argue that this is due to poor implementation, but never the less, it happens.
When this does happen, under normal circumstances (CA TCP task at priority 181),
the CA tasks sending notification to clients that the IOC is alive become
starved and the client eventually disconnects.  The client then tries to
reconnect, but is blocked because the CA TCP task is also starved.  At
this point someone usually notices, logs into the IOC, and finds the offending
task starving all tasks with lower priority.  This is fairly easy to diagnose.
When the CA TCP task has an elevated priority of say 54 (as was the case at
Jefferson Lab for some time), these reconnects are processed into new
CA client and event tasks which only die on the vine themselves because
of CPU starvation.  New connections consume file descriptors.  This
disconnect/reconnect process repeats until all the IOC file descriptors
are consumed and the IOC locks up and will not even allow telnet to connect.
This becomes a harder problem to diagnose as the IOC appears to hang for an
unknown reason and no diagnostic can be performed.  With the use of our
CA name server, we now have acceptable connection times without elevating
priorities.

-- 
Richard Dickson
Thomas Jefferson National Accelerator Facility
[email protected]
(757)269-5082

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System