Experimental Physics and
Industrial Control System

[email protected] (Jeff Hill) · Thu, 4 Jun 1998 10:18:32 -0600

Bill,

Sorry to copy you message to teck-talk, but it appears that some of this would
be interesting reading for the rest of the collaboration.

On Wednesday, June 03, 1998 10:44 PM, Bill Cruise [SMTP:[email protected]] wrote:
> Jeff Hill wrote:
> 
> > On Wednesday, June 03, 1998 4:49 PM, William L. Cruise [SMTP:[email protected]] wrote:
> > > We are trying to rebuild our Channel Access to TCL/TK interface.
> > > We have spent over a week trying to do this, with no success.  We
> > > really need to be able to rebuild this, as we feel the current
> > > HP-UX channel access library may solve some longstanding CA problems.
> > >
> >
> > What is the nature of your longstanding problems with CA?
> 
> Ever since we first started with EPICS we have had very problematicperformance with CA
> connections.  By this I mean that with medm,
> a very generic client, we would have some channels not connect
> when a form was opened or the IOC (we have onlly 1) rebooted.  The
> level of this is that a 40 channel form would experience some channels
> not connected about 1 time out of 5.
> 

Most problems with CA connection performance can be traced to the following
special situations. I think that sites that are aware of the following issues and have
not made local modifications to the CA source code will agree that CA connection
performance has been flawless.

1)
There are some complex integration issues between X and Channel Access when
there is only one thread of control. If a code isn't parked in ca_pend_event() and
is in say XtNextEvent() instead then it is necessary to poll Channel Access via
ca_pend-event(1e-12) at some rate (say every 100 mS) so that Channel Access
can check to see if its internal timers have expired and take care of its background 
activities. These background activities include sending search requests when
attempting to connect channels, and detecting that network connections have
gone down when channels are connected.

It is possible that you are having connection performance problems with medm
because medm is not polling ca_pend_event() every 100 mS or so. The medm code 
does call ca_pend_event() when there is activity on the CA socket file descriptors,
but it turns out that this will not always take care of internal timers inside of the ca client library.
The current author, Ken Evans, was (is) convinced that polling the ca client library
was (is) unnecessary because he has seen adequate connection performance at
his site. However there are many IOCs at his site with many UDP based server beacons so
it is possible that periodic UDP file descriptor activity causes ca_pend_event() to be
called at a reasonable rate. However, if a site has very few IOCs or if the network
is disconnected from medm's workstation then perhaps ca_pend_event() will no longer be
called at an acceptable rate because the workstation will not see enough (any) beacons.

2) 
Many hosts on your subnet are configured with the wrong netmask. These rogue hosts
respond to all CA search broadcasts with ICMP messages indicating that the CA search broadcast
was generated on the wrong subnet. If many machines are incorrectly configured in this
way the resulting ICMP flood can greatly reduce or shutdown CA's attempts to connect channels.

3)
CPU loads in your IOC do not allow the CA server to run. The CA server is the lowest
priority activity in your IOC and will be the first functionality to freeze when the CPU saturates.
This guarantees that essential internal IOC activity maintained in the scan tasks will not be interrupted
when the external network load imposed by the CA server is high.

> We were led to believe that this was normal performance for Channel
> Access, 

it isn't

> With et_wish we got (I believe) all the channels connecting, but with
> reboots of the IOC most of the channels would not reconnect.  

Make certain that a CA repeater is running on your workstation.
CA will start it for you automatically if it is in your path. Make certain that
the workstation is able to see the IOC's beacons (should always be the
case if the workstation and the IOC are on the same subnet - otherwise
see the CA reference manual).

> We were able to make a comparison with an medm recently built for
> Solaris (remember, we are a HPUX and sparc IOC shop!).  The Solaris
> medm worked perfectly.  We then rebuilt medm for HPUX.  The rebuilt
> medm connects perfectly, both originally and on reboots.  We then
> assumed that CA had changed significantly between when our utilities
> were built and presently.  A few checks showed that none of the
> utilities had been rebuilt when we upgrated to R3.13.b11.  Therefore
> we set out to build either et_wish or some other TCL/CA interface.
> 

Connection performance for large numbers of channels has been 
improved in 3.13, but I think that sites that are aware of the above 3 
issues will agree that connection performance under 3.12.2 has been 
flawless. Or perhaps there is a problem with some versions of HPUX 
(perhaps when you rebuilt you linked with a newer version of the OS 
libraries)?

> > > We currently run a version of "et_wish" which was built long ago,
> > > and which uses R3.12.0 channel access libraries.  It was built
> > > with TCL 7.6/4.2.  We have not been able to built a working
> > > version of this program using R3.13.0.beta11 libraries.
> > >
> >
> > I am not sure if this is causing your troubles, but I should mention that
> > there have been problems with compiling  traditional C based CA clients
> > (clients that do not use compiler options that result in
> > __STDC__ being defined and therefore disable function prototypes) under
> > R3.13.0.beta11 because I inadvertently ifdef'd out the CA function alias
> > macros in that version.
> >
> > Beta 12 will resolve this problem.
> >
> > it is possible to test to see if __STDC__ is defined by inserting this code:
> >
> > #ifdef __STDC__
> > #error yes, __STDC__ is defined
> > endif
> 
> I didn't try this for a couple of reasons.  One is that all the et_wish stuff iswritten in
> K&R, with no prototypes.  It should be right at home with __STDC__.

Typically traditional K&R C with no function prototypes is compiled with options
that do not result in __STDC__ being defined. You may find that the CA header
files in beta 12 are easier to use with traditional K&R C code.

Jeff

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System