EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <20062007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <20062007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: gateway enum writes
From: "Kenneth Evans, Jr." <[email protected]>
To: "'Jeff Hill'" <[email protected]>, "'Dirk Zimoch'" <[email protected]>, "'Leicester, PJ \(Pete\)'" <[email protected]>
Cc: "'Tim Mooney'" <[email protected]>, "'Ralph Lange \(BESSY\)'" <[email protected]>, <[email protected]>, "'Ned Arnold'" <[email protected]>, "'Andrew Johnson'" <[email protected]>
Date: Thu, 30 Mar 2006 08:25:11 -0600
All,

     First I apologize for not giving this my full attention.  I have been
out of town, the Gateway is no longer my primary responsibility, and there
are only so many hours in the day.

     On looking at the original message I can guess that you are using a
Joerger scaler.  The Joerger scaler does not play well with channel access.
It does not work well with MEDM, either on Solaris or Linux.  I investigated
this some time ago, to the extent I was convinced it was not MEDM.  The beam
line people that were using it at that time did not have enough expertise to
determine the real cause of the problem.  The same problem has appeared at
various sites since.

     Puts to the Gateway come from the Gateway's clients (EDM in this case).
Straight puts go through, puts with callback are queued.  (I have not looked
at the code again, as Jeff stated.)  I would expect EDM is doing a put with
callback.  I tested yesterday and had no trouble putting enum values such as
SCAN rates through the Gateway.  Moreover, this is done here on a regular,
daily basis without problems, and has been tested extensively in the past.

     My recollection is that the workaround for the Joerger scaler is to
push the buttons alternately to get it to be done.  You can also have two
screens and use the buttons on both of them.  You'll have to experiment.

     Thus I think the problem is with the instrument rather than the
Gateway.  You will probably have problems without a Gateway.  Having said
this, it should not be giving Virtual circuit unresponsive.  VCR's have been
an ongoing problems with CA since they were introduced.  (On the positive
side, they solve some significant problems.)  I made a fix to some of the
VCR problems, which I tested.  Jeff fixed it in another way for 3.14.8,
which I have not tested, but I have no evidence here that Jeff's fix is not
working, and I would expect it to be.  Note that it is the client that
decides it has a VCR.  In many cases there was nothing wrong with the server
(Gateway in this case).

	-Ken

-----Original Message-----
From: Jeff Hill [mailto:[email protected]] 
Sent: Wednesday, March 29, 2006 11:36 AM
To: 'Kenneth Evans, Jr.'; 'Dirk Zimoch'; 'Leicester, PJ (Pete)'
Cc: 'Tim Mooney'; 'Ralph Lange (BESSY)'; [email protected]
Subject: RE: gateway enum writes


Ok, so where are we. 

I think I understand that:

1) Strange behavior occurs only when the GW is attached to a PV that takes a
long time to complete a put callback.

2) Ken looked in the GW source code and verified that a put callback request
will postponed, when a put callback for that PV is already in progress,
until that put callback in progress completes.

Ken, perhaps you attempt to reproduce this issue against a record of this
type (with a very long put callback delay) at the APS?

Jeff

> -----Original Message-----
> From: Kenneth Evans, Jr. [mailto:[email protected]]
> Sent: Wednesday, March 29, 2006 9:04 AM
> To: 'Jeff Hill'; 'Dirk Zimoch'; 'Leicester, PJ (Pete)'
> Cc: 'Tim Mooney'; [email protected]
> Subject: RE: gateway enum writes
> 
> > One could speculate that the original GW design might have intended to
> > perform all puts with completion notification in order to know exactly
> when
> > to update the GW's cache.
> 
> > One could further speculate that a solution, for the GW, might be to not
> > start a write for a channel if a write is already outstanding on that
> same
> > channel. Instead the GW might save the value from the last write request
> for
> > a busy channel and initiate another write request using this saved value
> as
> > soon as the outstanding write request completes.
> 
>      That is what it does.
> 
> 	-Ken
> 
> -----Original Message-----
> From: Jeff Hill [mailto:[email protected]]
> Sent: Wednesday, March 29, 2006 9:57 AM
> To: 'Dirk Zimoch'; 'Leicester, PJ (Pete)'
> Cc: 'Tim Mooney'; [email protected]; 'Ken Evans'
> Subject: RE: gateway enum writes
> 
> 
> Here my two centi-euro's worth.
> 
> > if (ctx.getMsg()->m_cmmd == CA_PROTO_WRITE_NOTIFY) {
> >    docallback == GATE_DOCALLBACK;
> > } else {
> >    docallback == GATE_NOCALLBACK;
> > }
> 
> The context (in this case the variable is named ctx) is a class whose
> definition is private to the portable server and subject to change in the
> future. Its header file is typically not installed for public consumption.
> We would not want, for example, to have the gateway code become too
> dependent on the internal details of the server or its protocol. That
> might
> create maintenance headaches.
> 
> One could speculate that the original GW design might have intended to
> perform all puts with completion notification in order to know exactly
> when
> to update the GW's cache.
> 
> Nevertheless, use of put callback just might cause behavior problems for
> the
> gateway if the destination record takes a long time to complete the write
> request, and a 2nd write is started while the GW has a write to the same
> record in progress. That would block the incoming protocol stream from the
> GW to the IOC (for all channels on that IOC). The CA server is unable to
> initiate another put notify request with this record until it completes
> the
> put notify request that is pending, and in that situation the current
> server
> design suspends processing of incoming requests until it can initiate the
> put notify request (when the record completes processing).
> 
> One could further speculate that a solution, for the GW, might be to not
> start a write for a channel if a write is already outstanding on that same
> channel. Instead the GW might save the value from the last write request
> for
> a busy channel and initiate another write request using this saved value
> as
> soon as the outstanding write request completes.
> 
> Jeff
> 
> > -----Original Message-----
> > From: Dirk Zimoch [mailto:[email protected]]
> > Sent: Wednesday, March 29, 2006 4:41 AM
> > To: Leicester, PJ (Pete)
> > Cc: Tim Mooney; [email protected]; Ken Evans
> > Subject: Re: gateway enum writes
> >
> > This is from the gateway code (gateVc.cc):
> >
> > caStatus gateVcData::write(const casCtx& ctx, const gdd& dd, gateChan
> > &/*chan*/)
> > {
> > 	int docallback=GATE_DOCALLBACK;
> >
> > [...]
> > 	switch(at) {
> > [...]
> > 	case gddAppType_ackt:
> > 	case gddAppType_acks:
> > 		docallback = GATE_NOCALLBACK;
> > 		// Fall through
> > 	default:
> > [...]
> > 			caStatus stat = pv->put(&dd, docallback);
> > 			if(stat != S_casApp_success) return stat;
> >
> > 			if(docallback) {
> > 				// Start a pending write
> > #if DEBUG_GDD
> > 				fflush(stderr);
> > 				printf("pending_write\n");
> > 				fflush(stdout);
> > #endif
> > 				pending_write = new
> gatePendingWrite(*this,ctx,dd);
> > 				if(!pending_write) return S_casApp_noMemory;
> > 				else return S_casApp_asyncCompletion;
> > 			} else {
> > 				return S_casApp_success;
> > 			}
> >
> > It seems that ALL puts except alarm handler ackt and acks are done with
> > callback. There is no direct flag to write() from the generic CA server
> > which indicates writes with or without callback, as far as I can see.
> > But the gateway should have look at the CA command in ctx, probably like
> > this:
> >
> > if (ctx.getMsg()->m_cmmd == CA_PROTO_WRITE_NOTIFY) {
> >    docallback == GATE_DOCALLBACK;
> > } else {
> >    docallback == GATE_NOCALLBACK;
> > }
> >
> >
> > Dirk
> >
> >
> > Leicester, PJ (Pete) wrote:
> > > I can confirm that the problem appears isolated to the scaler record
> CNT
> > > field and not enums in general.
> > >
> > > It looks like Tim's ca_put_callback theory may be correct. Is there a
> > > Gateway expert out there who can confirm this? How are writes done
> from
> > > the Gateway. Does it always use ca_put_callback regardless of how the
> > > put reached the gateway?
> > >
> > > Thanks for everyones responses on this.
> > >
> > > Pete Leicester
> > > Diamond
> > >
> > > -----Original Message-----
> > > From: Tim Mooney [mailto:[email protected]]
> > > Sent: 28 March 2006 21:07
> > > To: Leicester, PJ (Pete)
> > > Cc: [email protected]
> > > Subject: Re: gateway enum writes
> > >
> > >
> > > Leicester, PJ (Pete) wrote:
> > >
> > >>I am getting some strange behaviour when writing enumerations through
> > >>the Gateway (version 2.0.0.0 on 3.14.8.2 and RedHat Enterprise 4).
> > >>
> > >>The problem first showed itself in edm when pressing a button on a edm
> > >>screen to send the value resulted in the following error:
> > >>
> > >>    CA.Client.Exception...............................................
> > >>        Warning: "Virtual circuit unresponsive"
> > >>        Context: "diamrs0005l.diamond.ac.uk:6064"
> > >>        Source File: ../tcpiiu.cpp line 896
> > >>        Current Time: Tue Mar 28 2006 16:19:01.334736000
> > >>    ..................................................................
> > >>
> > >>I did some further tests using caput with similar results:
> > >>
> > >>    [pjl45@pc0005 pjl45]$ caput  -w15 GDA:scaler2.CNT Count
> > >>    Old : GDA:scaler2.CNT                Done
> > >>    Read operation timed out: PV data was not read.
> > >>    New : GDA:scaler2.CNT
> > >>    CA.Client.Exception...............................................
> > >>        Warning: "Virtual circuit disconnect"
> > >>        Context: "op=0, channel=GDA:scaler2.CNT, type=DBR_TIME_STRING,
> > >>count=1, ctx="diamrs0005l.diamond.ac.uk:6064""
> > >>        Source File: ../getCopy.cpp line 82
> > >>        Current Time: Tue Mar 28 2006 17:04:24.488496000
> > >>.
> > >>Despite the above error message the write does actually reach the IOC.
> > >>However if I now try the change the value back as follows:
> > >>
> > >>    [pjl45@pc0005 pjl45]$ caput  -w15 GDA:scaler2.CNT Done
> > >>    Old : GDA:scaler2.CNT                Count
> > >>    Read operation timed out: PV data was not read.
> > >>    New : GDA:scaler2.CNT
> > >>    CA.Client.Exception...............................................
> > >>        Warning: "Virtual circuit disconnect"
> > >>        Context: "op=0, channel=GDA:scaler2.CNT, type=DBR_TIME_STRING,
> > >>count=1, ctx="diamrs0005l.diamond.ac.uk:6064""
> > >>        Source File: ../getCopy.cpp line 82
> > >>        Current Time: Tue Mar 28 2006 17:22:36.167980000
> > >>    ..................................................................
> > >>
> > >>I get an error again however this time the 'Done' value gets written
> > >>to
> > >>the IOC exactly ONE MINUTE after I entered the caput command. This is
> > >>long after the caput command has timed out so it appears the gateway
> > >
> > > is
> > >
> > >>responsible for the delay?
> > >>
> > >>Has anyone any idea what may be happening? Is there a 60 second
> > >>timeout
> > >>in CA or the gateway which may give a clue as to what I am seeing?
> > >>
> > >>(For the record this test was done with a very lightly loaded test
> > >>gateway serving only 20 or so PV's. Also using caput -w70 results in
> > >
> > > the
> > >
> > >>same timeouts)
> > >
> > >
> > > Do you get this kind of result with *all* enum fields that you write
> to
> > > through the gateway, or only this one?  The scaler record's CNT field
> is
> > > different from other enum's in that it starts an operation that may
> take
> > > a long time to complete.  I don't know what the gateway is using to do
> > > put's, but if it should happen to be ca_put_callback(), the gateway
> may
> > > be waiting for a callback that will come only after the scaler has
> > > finished counting.
> > >
> >
> > --
> > Dr. Dirk Zimoch
> > Swiss Light Source
> > Paul Scherrer Institut
> > Computing and Controls
> > phone +41 56 310 5182
> > fax   +41 56 310 4413




Replies:
Re: gateway enum writes D. Peter Siddons
Re: gateway enum writes Kate Feng
References:
RE: gateway enum writes Jeff Hill

Navigate by Date:
Prev: Re: stringin/out <-> sub record ? Martin L. Smith
Next: Re: gateway enum writes D. Peter Siddons
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <20062007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: gateway enum writes Jeff Hill
Next: Re: gateway enum writes D. Peter Siddons
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  <20062007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 02 Sep 2010 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·