Experimental Physics and
Industrial Control System

[email protected] (Ned Arnold) · Wed, 19 Feb 1997 08:42:51 -0600

> We (SLAC's PEP-II RF group) are having a problem with hanging/crashing 
> IOCs and wonder if anyone has some suggestions.  We're running EPICS 
> R3.13.0.beta4 with dm version 2.3.  The processor is a National 
> Instruments VXIcpu-030 with 8 MB of RAM.  When everything is up and 
> running normally we see about 2 MB of free memory.  Task stacks are not 
> close to the edge.  There is no VXI interrupt activity, although perhaps
> some are being genterated by the Allen Bradley scanner module.
> 
> Occasionally we see the Allen Bradley driver spontaneously changing the 
> status of all connected links from good to bad.  Sometimes they go back 
> to good a moment later, sometimes it's a long while later, sometimes not 
> at all.  There is nothing an operator can do to cause this directly.  
> The blue hose is not being unplugged or anything like that.  We're
> using a 6008-SV1 scanner module.

Since you are in a VXI crate, you must be using some type of VME/VXI adapter
for the 6008. I remember spending WEEKS of time several years ago debugging
a similar problem and discovering that my VME/VXI adapter did not 
handle the "read-modify-write" cycle correctly (this cycle is used to provide
mutual exclusion to the mailbox memory on the 6008). My symptoms were very
similar. After receiving some "updated PLD's" from the manufacturer, I have
not had a problem. I am using the older version of the 6008 with a
CDS73A-851.

> One possibility I wonder about is that CA clients cause the creation 
> of "CA client" and "CA event" tasks that consume a lot (800 Mb) of 
> memory.  Perhaps these crashes are due to one too many dm sessions
> being fired up in people's offices.  Is there a way to limit the
> number of CA clients?  If not, shouldn't such a thing be added?
> What do these tasks do with that much memory?  When IOC resources 
> run out, does CA currently stop allowing new client connections or 
> does it let the IOC die?  How do other people handle this situation?  
> Just add more memory?

It has been my experience that when an ioc runs out of memory so that no
new clients can connect, the ioc does not crash, but merely refuses new
connections. Looking at ALL your symptoms, I don't think your problem is
a lack of memory resources (at least not your most important problem.)

	Ned

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System