Experimental Physics and
Industrial Control System

"Jeff Hill" <[email protected]> · Thu, 8 Nov 2007 13:26:46 -0700

A feature of R3.14 is that if the channel changes IP addresses and the old
server doesn't execute an orderly shutdown then the R3.14 client library
will disconnect the application, but wait the full duration of the TCP IP
keep-alive interval before it abandons the TCP circuit and attempts to
reconnect. 

This "feature" greatly improves the stability of EPICS systems experiencing
load saturation.

You might experience this delayed connect behavior if for example you turn
off a vxWorks or RTEMS (an embedded) IOC (w/o executing an orderly
shutdown), and then boot the system specifying that is to use a different IP
address.

In R3.14 we have not yet implemented an orderly shutdown for the IOC's CA
server, the IOC's CA database links, and the IOC's database. However, on a
process oriented OS like Linux, when the IOC's process is shutdown, Linux
will execute the orderly shutdown of the TCP circuits even if the IOC is
negligent.

PS: Just for clarification, I should add that orderly shutdown in the IOC
for the CA components is currently somewhat blocked by lack of orderly
shutdown for the database because the CA server and client are unable to
clean up their event queues until the database stops posting to them. 

PPS: This "feature" might also cause the client switchover to a redundant
IOC to be slower - when the switchover is due to a catastrophic software or
hardware failure. One possible upgrade might be to search over UDP for
channels attached to an unresponsive circuit at a very slow rate, and
abandon the unresponsive circuit only if the IP address of one of its
channels is observed to change. However, that change would somewhat dilute
the benefits of the current intended behavior which is to postpone adding
additional load when a congestion is observed. Another option might be to
configure the disconnect timeout for unresponsive circuits independent of
both the TCP keep-alive timer and EPICS_CA_CONN_TMO (users choose how long
they will wait to recover from congestion before they will attempt to
reconnect, and insert additional load).

Jeff

> -----Original Message-----
> From: [email protected] [mailto:tech-talk-
> [email protected]] On Behalf Of Brian Bevins
> Sent: Thursday, November 08, 2007 11:17 AM
> To: Szalata, Zenon M.
> Cc: EPICS Techtalk
> Subject: Re: EDM screens not updating on Linux
> 
> Hi Zen,
> 
> That sounds more like a channel access configuration problem. Is your
> IOC correctly set up to send server beacons to the machine where you're
> running EDM? See:
> 
> http://www.aps.anl.gov/epics/base/R3-14/9-docs/CAref.html#Configurin2
> 
> --Brian
> 
> Szalata, Zenon M. wrote:
> > I have seen a similar behaviour in EDM.  The most recent was: EDM was
> > running and was displaying correctly data from an IOC.  I restarted
> > the IOC, the EDM displays went white, as they should, but never
> > showed that the EDM reconnected untill I placed it in edit then
> > immediately in execute mode.  It was not the IOC because for a few
> > minutes I was monitoring one of the PV's with a caget; the IOC was
> > publishing its data on the CA long before (minutes) I made EDM
> > recover.  I am using edm version 1-11-0zb running on RHEL4. Zen
> >
> > -----Original Message-----
> > From: [email protected]
> > [mailto:[email protected]] On Behalf Of Dirk Zimoch
> > Sent: Thursday, November 08, 2007 1:05 AM
> > To: Brian Bevins Cc: EPICS
> > Techtalk Subject: Re: EDM screens not updating on Linux
> >
> > Hi Brian,
> >
> > we have seen the same effect with MEDM. I suspect a problem with
> > either the X-server or the motif library. It is very strange that the
> > effect appears only on some computers.
> >
> > It seems that the screens don't get update requests when new values
> > arrive. Minimizing and maximizing, or just "wiping" the screen with
> > some other window generates the update events and the current value
> > is shown. From this I conclude that there is no problem in EPICS and
> > the monitor events correctly arrive. The values are just not updated
> > on the screen. Can you verify this with EDM?
> >
> > Dirk
> >
> > Brian Bevins wrote:
> >> We are in the process of converting our control room machines from
> >>  HP-UX 11.11 to RedHat Enterprise Linux (rhel) 4WS. Most things
> >> seem to be working fine, but an odd problem has cropped up with EDM
> >> that we did not have on HP-UX.
> >>
> >> Every now and then an EDM display will simply stop showing any
> >> updates to the PV's. Closing and re-opening the display gets it
> >> working again. Writes from the screen to the ioc's seem to work
> >> normally.
> >>
> >> Has anybody else experienced a problem like this?
> >>
> >> We are using EDM 1-10-2 on rhel 4WS (Nahant Update 5). This is the
> >>  same version we have bee using on HP-UX with no problems.
> >>
> >> Our operators love how zippy EDM runs on their new Linux boxes
> >> compared to the ancient HP hardware, but this is really denting
> >> their enthusiasm.
> >>
> >> Thanks, --Brian
> >>
> >
> > -- Dr. Dirk Zimoch Paul Scherrer Institut, WBGB/006 5232 Villigen
> > PSI, Switzerland Phone +41 56 310 5182
> >
> 
> 
> --
> Brian S. Bevins, PE
> Computer Scientist / Mechanical Engineer
> Thomas Jefferson National Accelerator Facility
> 
>       "Progress, far from consisting in change,
>        depends on retentiveness. Those who cannot
>        remember the past are condemned to repeat it."
>                                           -- George Santayana

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System