Experimental Physics and
Industrial Control System

Hi Bruce,

 

Ø  17_14:39:08:CAS: UDP recv error (errno=Interrupted system call)

 

Revision 11734, occurring on 2009-5-8, made by Andrew to cast_server.c suppresses that message. However neither the previous version nor the current version use CPU when this happens (there is a sleep call when the error is handled). I am curious about the surrounding circumstances that were precipitating that mod as signals are typically not delivered to this thread.

 

Which OS is it running on? Interrupted system calls typically occur on OS with signals such as UNIX. Most UNIX systems now dispatch some of the signals per-thread and other ones are per-process. There is some limited support for signals on vxWorks as I recall.

 

Do other networking capabilities on the IOC such ping, telnet, NFS, or CA continue to work when this is happening?

 

Hard to make further guess with this info.

 

Jeff
______________________________________________________
Jeffrey O. Hill           Email        [email protected]
LANL MS H820              Voice        505 665 1831
Los Alamos NM 87545 USA   FAX          505 665 5107

 

Message content: TSPA

 

With sufficient thrust, pigs fly just fine. However, this is

not necessarily a good idea. It is hard to be sure where they

are going to land, and it could be dangerous sitting under them

as they fly overhead. -- RFC 1925

 

From: [email protected] [mailto:[email protected]] On Behalf Of Bruce Hill
Sent: Thursday, February 17, 2011 4:32 PM
To: [email protected]
Cc: Stoffel, Pavel
Subject: Fwd: Re: GigE cameras stops

 

We're having a problem with 'CAS: UDP recv errors" that's causing
our GigE camera IOC's to stop updating images overnight.

We're running EPICS 3.14.9, asyn R4.10.1, and areaDetector R1-2.

When the images stop, we get these errors on our console:

17_14:39:08:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:08:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:09:CAS: UDP recv error (errno=Interrupted system call)

...

Googling "Interrupted system call" seems to indicated that repeating the
system call, in this case, recv_from, may be needed.   It appears that the CA
code is already doing that.

I'm hoping that Jeff or Mark, or anyone, may have already seen this problem.
We've been working on upgrading to 3.14.11 and newer modules, so if this
problem has already been fixed we may just need to refocus our efforts on
the upgrade.

Thanks,
- Bruce


-------- Original Message --------

Subject:	Re: GigE cameras stops
Date:	Thu, 17 Feb 2011 14:45:52 -0800
From:	Pavel Stoffel <[email protected]>
To:	Hill, Bruce <[email protected]>

Hi Bruce,

Turned on tracing for cameras:

It looks like the camera stops working when we get these 'CAS: UDP recv 

errors (errno=Interrupted system call)'.

Everything before it, repeats for every frame.  In this case, I 

collected 54335 frames before it stopped.

Here is the tail end of the trace file:

17_14:38:22:2011/02/17 14:38:22.888 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45326

17_14:38:23:2011/02/17 14:38:23.088 prosilica:frameCallback: 

frameId=45329, timeStamp=763787.037673

17_14:38:23:2011/02/17 14:38:23.088 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45327

17_14:38:23:2011/02/17 14:38:23.288 prosilica:frameCallback: 

frameId=45330, timeStamp=763787.237673

17_14:38:23:2011/02/17 14:38:23.288 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45328

17_14:38:23:2011/02/17 14:38:23.488 prosilica:frameCallback: 

frameId=45331, timeStamp=763787.437673

17_14:38:23:2011/02/17 14:38:23.488 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45329

17_14:38:23:2011/02/17 14:38:23.688 prosilica:frameCallback: 

frameId=45332, timeStamp=763787.637673

17_14:38:23:2011/02/17 14:38:23.688 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45330

17_14:38:23:2011/02/17 14:38:23.888 prosilica:frameCallback: 

frameId=45333, timeStamp=763787.837673

17_14:38:23:2011/02/17 14:38:23.888 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45331

17_14:38:24:2011/02/17 14:38:24.088 prosilica:frameCallback: 

frameId=45334, timeStamp=763788.037673

17_14:38:24:2011/02/17 14:38:24.088 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45332

17_14:38:24:2011/02/17 14:38:24.288 prosilica:frameCallback: 

frameId=45335, timeStamp=763788.237673

17_14:38:24:2011/02/17 14:38:24.288 XPP:USR:GIGE1:cam1:ImageCounter_RBV 

devAsynInt32::interruptCallbackInput new value=45333

17_14:38:24:2011/02/17 14:38:24.456 PS1 addr 0 queueRequest priority 0 

not lockHolder

17_14:38:24:2011/02/17 14:38:24.456 PS1 callback

17_14:39:08:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:08:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:09:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:09:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:10:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:17:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:17:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:18:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:18:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:19:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:32:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:33:CAS: UDP recv error (errno=Interrupted system call)

17_14:39:55:CAS: UDP recv error (errno=Interrupted system call)

Pavel Stoffel wrote:

> Hi Bruce,

> 

> The cameras stopped after about 30-40K frames.  IOC 1 had 104 frame 

> errors, IOC 2 1:

> 

> $ ls-cam1

> XPP:USR:GIGE1:cam1:ImageRate_RBV 0

> XPP:USR:GIGE1:cam1:ImageCounter_RBV 42931

> XPP:USR:GIGE1:cam1:PSBadFrameCounter_RBV 104

> 

> $ ls-cam2

> XPP:USR:GIGE2:cam1:ImageRate_RBV 0

> XPP:USR:GIGE2:cam1:ImageCounter_RBV 29353

> XPP:USR:GIGE2:cam1:PSBadFrameCounter_RBV 1

> 

> Don't see any errors in the log file :-(

> 

> Thanks,

> Pavel

> 

> 

>   

-- 
Bruce Hill
Member Technical Staff
SLAC National Accelerator Lab
2575 Sand Hill Road M/S 10
Menlo Park, CA  94025