Hi Mitch,
thanks a lot for finding and fixing this.
I slightly modified your patch (increased flushTime and used a fixed
timeout instead of epicsThreadSleepQuantum)
I will do some tests before commiting this.
Jens
_______________________________
Jens Eden
Physikalisch-Technische Bundesanstalt
Albert-Einstein-Straße 15
12489 Berlin
Tel +49 30 3481-7160
Von: "D'Ewart, J. Mitch" <[email protected]>
An: Jens Eden <[email protected]>,
Kopie: "[email protected]" <[email protected]>
Datum: 11.03.2014 17:42
Betreff: RE: MAXv model-3 - error reading axis position after 5
attempts
Hi Jens,
Thanks for looking at this.
> Currently timeout is set to: timeout = epicsThreadSleepQuantum() +
0.001;
>
> Here this sums up to 21 msecs (epicsThreadSleepQuantum() = 20 msecs)
and I
> never got a timeout here.
In our case (mvme 6100 and RTEMS) epicsThreadSleepQuantum() = 0.001, so we
were only waiting 2 ms before timeout. I have increased it to 21ms.
> Could you send me your modifications?
Here is the patch - it does a wait after clearing the message semaphore
and then rechecks if any more messages have been sent. This also has the
hardcoded timeout of 0.021s.
--- motorApp/OmsAsynSrc/omsMAXv.cpp 2014-01-20 04:58:11.000000002
-0800
+++ motorApp/OmsAsynSrc/omsMAXv.cpp 2014-03-11 08:24:22.000000002
-0700
@@ -296,7 +296,8 @@
}
/* see if junk at input port - should not be any data available */
- if ((epicsUInt16) pmotor->inGetIndex != (epicsUInt16)
pmotor->inPutIndex)
+ int flushTime = 0;
+ while (((epicsUInt16) pmotor->inGetIndex != (epicsUInt16)
pmotor->inPutIndex) && (flushTime < 5))
{
// flush cards response Buffer
#ifdef DEBUG
@@ -311,7 +312,8 @@
flag1.All = pmotor->status1_flag.All;
pmotor->status1_flag.All = flag1.All;
pmotor->msg_semaphore=0;
-
+ epicsThreadSleep(epicsThreadSleepQuantum());
+ flushTime++;
}
putIndex = (epicsUInt16) pmotor->outPutIndex;
@@ -383,7 +385,7 @@
itera = 0;
double time = 0.0;
- double timeout = epicsThreadSleepQuantum() + 0.001;
+ double timeout = 0.021;
while ((pmotor->status1_flag.Bits.text_response == 0) && (time <
timeout)){
Debug(32, "%s:%s:%s: Waiting for reponse, itera:%d\n",
driverName, functionName, portName, itera);
________________________________________
From: Jens Eden [[email protected]]
Sent: Monday, March 10, 2014 8:04 AM
To: D'Ewart, J. Mitch
Cc: [email protected]
Subject: Re: MAXv model-3 - error reading axis position after 5 attempts
Mitch,
> It looks like the timeout may not be long enough so our messages are
getting out of sync.
Do you get timeout error messages, if setting "motorOMSBASEdebug = 1"?
> 1. increase the timeout time ...
Currently timeout is set to: timeout = epicsThreadSleepQuantum() + 0.001;
Here this sums up to 21 msecs (epicsThreadSleepQuantum() = 20 msecs) and I
never got a timeout here.
Since epicsThreadSleepQuantum may vary, it shouldn't be used to set the
timeout.
Could you try to set timeout to 21 msecs (timeout = 0.021) and see if that
is sufficient?
2. multiple responses in sendOnly().
I did this quicj test to reproduce this:
1. send "AY LT?", read response, send "WY" read response
caput OMSMv:io3401omsSendRecv "AY LT?" ; caput OMSMv:io3401omsSendRecv
"WY"
omsMAXv::send: sending: AY LT?
omsMAXv::sendReceive: received =l
omsMAXv::send: sending: WY
omsMAXv::sendReceive: received MAXv-8000 ver:1.45 ...
2. send "AY LT?", don't read response, send "WY" read response
caput OMSMv:io3401omsSend "AY LT?" ; caput OMSMv:io3401omsSendRecv "WY"
omsMAXv::send: sending: AY LT?
omsMAXv::send: sending: WY
omsMAXvAsyn:sendOnly:MAXv1: flushing 3 characters <= flush unread
response
omsMAXv::sendReceive: received MAXv-8000 ver:1.45 ...
3. send "AY LT? AX LT?", don't read 2 responses, send "WY" read response
caput OMSMv:io3401omsSend "AY LT? AX LT?" ; caput
OMSMv:io3401omsSendRecv "WY"
omsMAXv::send: sending: AY LT? AX LT?
omsMAXv::send: sending: WY
omsMAXvAsyn:sendOnly:MAXv1: flushing 3 characters <= flush first
unread response
omsMAXv::sendReceive: received =l <= 2nd unread
response, should be firmware string
4. send more than 2 requests: MAXv remains unresponsive until power
cycle:
caput OMSMv:io3401omsSend "AY LT?;AX LT?;AZ LT?;" ; caput
OMSMv:io3401omsSendRecv "WY"
> 2) Mitch, changed the function sendOnly() to flush multiple responses.
Could you send me your modifications?
Thanks
Jens
Am 07.03.2014 18:57, schrieb D'Ewart, J. Mitch:
> The problem we are seeing
>
> omsBaseDriver:getAxesArray:MAXv-01: array string conversion error,
count: 0, axes: 8, input: >MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN<^M
>
> This seems to be a request and response not matching up. In this case
the response to getAxesArray() should be numerical, for example
>
> "AM RP?"
> responds with
> "0,0,0,0,0,0,0"
> but is getting
> "MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN"
>
> In this case sendOnly() is called from sendReceive(). sendOnly()
flushes the response queue
>
> /* see if junk at input port - should not be any data available */
> if ((epicsUInt16) pmotor->inGetIndex != (epicsUInt16)
pmotor->inPutIndex)
> {
> // flush cards response Buffer
> #ifdef DEBUG
> int deltaIndex = ((epicsUInt16)pmotor->inPutIndex) -
((epicsUInt16)pmotor->inGetIndex);
> Debug(32, "%s:%s:%s: flushing %d characters\n",
> driverName, functionName, portName, (((deltaIndex < 0)
? BUFFER_SIZE +
> deltaIndex : deltaIndex)));
> #endif
> putIndex = (epicsUInt16) pmotor->inPutIndex;
> pmotor->inGetIndex = putIndex;
> pmotor->status1_flag.Bits.text_response = 0;
> flag1.All = pmotor->status1_flag.All;
> pmotor->status1_flag.All = flag1.All;
> pmotor->msg_semaphore=0;
>
> }
>
> and then sends the new request. sendReceive() then waits for a
response flag to be set or timeout.
>
> Debug(64, "omsMAXv::sendReceive: receiving\n");
> itera = 0;
> double time = 0.0;
> double timeout = epicsThreadSleepQuantum() + 0.001;
> while ((pmotor->status1_flag.Bits.text_response == 0) && (time <
timeout)){
> Debug(32, "%s:%s:%s: Waiting for reponse, itera:%d\n",
> driverName, functionName, portName, itera);
> // busy-waiting but not more than 2000 times
> if (itera > 2000){
> time += epicsThreadSleepQuantum();
> epicsThreadSleep(epicsThreadSleepQuantum());
> }
> itera++;
> }
>
> It looks like the timeout may not be long enough so our messages are
getting out of sync. The two proposed solutions are to 1. increase the
timeout time and 2. flush multiple responses in sendOnly().
>
> Mitch D'Ewart
>
>
>
> ________________________________________
> From: [email protected] [[email protected]] On
Behalf Of Mark Rivers [[email protected]]
> Sent: Thursday, March 06, 2014 3:33 PM
> To: Luchini, Kristi L.; [email protected]
> Subject: RE: MAXv model-3 - error reading axis position after 5
attempts
>
> When sendOnly() is called does the MaxV actually send a response to that
message? If so then sendOnly should wait for it, even if it will discard
the response. If it does not then the next time a thread does a
write/read it cannot reliably know if a response it got was from its query
or a previous sendOnly response.
>
> Mark
>
>
> -----Original Message-----
> From: [email protected] [
mailto:[email protected]] On Behalf Of Luchini, Kristi L.
> Sent: Thursday, March 06, 2014 5:29 PM
> To: [email protected]
> Subject: MAXv model-3 - error reading axis position after 5 attempts
>
> Hello,
>
> Has anyone seen this error on the new model-3 version of the MAXv driver
(omsAsyn) ?
>
> 8544 omsBaseDriver:BasePoller:MAXv-01: error reading axis position
after 5 attempts^M
> 8545 omsBaseDriver:BasePoller:MAXv-01: error reading axis position
after 5 attempts^M
> 8546 omsBaseDriver:getAxesArray:MAXv-01: array string conversion
error, count: 0, axes: 8, input:
>MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN<^M
> 8547 2014/02/05 12:03:30.540 omsBaseDriver:readInt32:MAXv-01 wrong
AUX state N^M
> 8548 2014/02/05 12:03:30.670omsBaseDriver:getAxesArray:MAXv-01:
array string conversion error, count: 0, axes: 8, input:
>MNNN,PNNN,MNNN,PNLN,PNLN,PNLN,PNLN,PNLN<^M
>
> We have seen this error in a few motion control applications with MAXv
controllers. All of the IOCs were
> running motor-R6-8 asyn MAXv driver. The problem seems to occur when
we're using a number of axis, and in
> some cases using virtual motors where multiple axis are moved at once.
>
> 1) MAXv controller, with 2 steppers and 1 servo motors. This MAXv has
FW 1.45
> 2) MAXv controller , with 8 steppers. This MAXv has FW 1.44 (vitual
motors moving 4-axis at once, and 2-axis at once using transform record)
> 3) MAXv controller, with 7 steppers. This MAXv has FW 1.31 (virtual
motors, moves 4 axis at once, and 2-axis using transform record)
>
>
> Mitch D'Ewart found that "The omsMAXv::sendOnly function checks the
input buffer and flushes one response if there is anything there but we
seem to have come on a case where there are multiple responses backed up.
These responses are controlled by a message semaphore. It seems that once
sendOnly flushes the input buffer and clears the semaphore then the MAXv
sends another backed up message response. Finally the receive part of the
calling function gets the wrong message.
>
> There are two solutions that have been implemented recently at SLAC.
Both will be booted on Monday 3/10/2014, in production.
> So we don't know if the problem is fixed yet.
> 1) Ziga added eieio in omsMAXv.cpp by including basicIoOps.h from Till
Strauman.
>
> Example:
> static void writeReg8(volatile epicsUInt8 *a8, epicsUInt8 value)
> {
> #ifdef HAS_IOOPS_H
> out_8(a8, value);
> #else
> *a8 = value;
> #endif
> }
>
> 2) Mitch, changed the function sendOnly() to flush multiple responses.
>
> Details:
> Motor-R6-8
> Asyn-R4-12
> EPICS R3.14.12
> RTEMS-4.9.4
> MVME6100 CPU.
> MAXV-8000
>
> QUESTION:
> How shold asyn support implementation respond in this sort of case,
where the response messages get out of sync with the queries?
> Do you simply clear you the response buffer, buffer and start again, or
is there another solution? Has anyone else implemented a fix to this
> Particular MAxv but or to another asyn support driver?
>
> Regards,
> Kristi
>
>
>
>
>
- References:
- MAXv model-3 - error reading axis position after 5 attempts Luchini, Kristi L.
- RE: MAXv model-3 - error reading axis position after 5 attempts Mark Rivers
- RE: MAXv model-3 - error reading axis position after 5 attempts D'Ewart, J. Mitch
- Re: MAXv model-3 - error reading axis position after 5 attempts Jens Eden
- RE: MAXv model-3 - error reading axis position after 5 attempts D'Ewart, J. Mitch
- Navigate by Date:
- Prev:
RE: MAXv model-3 driver - omsMAXvConfig problem Jens . Eden
- Next:
Re: Problem with generating and installing OS specific header Andrew Johnson
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
<2014>
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
RE: MAXv model-3 - error reading axis position after 5 attempts D'Ewart, J. Mitch
- Next:
Re: MAXv model-3 - error reading axis position after 5 attempts Benjamin Franksen
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
<2014>
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|
·
Home
·
News
·
About
·
Base
·
Modules
·
Extensions
·
Distributions
·
Download
·