EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: CA gateway dies without error message
From: Dirk Zimoch <[email protected]>
To: Jeff Hill <[email protected]>
Cc: [email protected]
Date: Fri, 20 Aug 2010 12:09:56 +0200
Hi Jeff,

There is nothing to trace. The process is not there any more. The gateway process just ends without any message. Just as if something has called exit().

My start line is:
export EPICS_CA_MAX_ARRAY_BYTES=4000000
ulimit -s 2048
export EPICS_CAS_BEACON_ADDR_LIST=172.21.10.255
caGateway -no_cache -archive -prefix FIN-CCAGW11W-EXP \
	-sip 172.21.10.22 \
	-cip 172.21.11.255 \
	-uid 922 -gid 922 \
	-pvlist GATEWAY.pvlist \
	-access GATEWAY.access \
	-log gateway.log \
	"$@" > gateway.err 2>&1 < /dev/null &

Gateway startup log:
Aug 19 15:32:36 PV Gateway Version 2.0.4.0 [Oct 16 2009 10:54:59]
EPICS 3.14.11 PID=4757
EPICS_CA_ADDR_LIST=172.21.11.255
EPICS_CA_AUTO_ADDR_LIST=NO
EPICS_CA_SERVER_PORT=Not specified
EPICS_CA_MAX_ARRAY_BYTES=4000000
EPICS_CAS_INTF_ADDR_LIST=172.21.10.22
EPICS_CAS_SERVER_PORT=Not specified
EPICS_CAS_IGNORE_ADDR_LIST=Not specified
Running as user cagw on host fin-ccagw11w
Statistics PV prefix is FIN-CCAGW11W-EXP

There is nothing in gateway.err and no error message in gateway.log with a time stamp near the crash. The usual goodbye message like "Jul 13 17:12:40 PV Gateway Ending (SIGTERM)" is missing.

I will try to enable core dumps. Haven't done that yet.

Dirk

Jeff Hill wrote:
Hi Dirk,

I did put some considerable effort into improving the quality (in terms of
efficiency, stability, performance, and functionality) of the PCAS and the gateway around this time last year. And I believe that the result was a
significant step forward. Which version of the gateway are you running?


Currently, I have no clues on your issue to start with so it's hard to move forward. So I will 2nd Andrew's suggestion of attaching gdb and getting a
stack trace of the failed thread. One can attach gdb to a running process by
typing
"gdb <full path to image> <process id>". If gdb can't be attached to the running process then another alternative is to attach gdb to the core
file using "gdb <full path to image> <full path to core file>".
In gdb type first "bt" to show the thread that crashed and then follow that with "thread apply all bt" so that I know what all threads are up to when the crash occurred. Lots of output
but exactly what is needed to get the matter resolved. This works better if the code is compiled for debugging, but I might be able to resolve the issue
with only the stack traces from the optimized version (which runs faster and so
can have a different set of failure scenarios) - so I am very willing to
look at traces from an optimized version first if it will save some effort on your end.


Thanks in advance for your help,

Jeff
______________________________________________________
Jeffrey O. Hill           Email        [email protected]
LANL MS H820              Voice        505 665 1831
Los Alamos NM 87545 USA   FAX          505 665 5107

Message content: TSPA


-----Original Message-----
From: [email protected] [mailto:tech-talk-
[email protected]] On Behalf Of Dirk Zimoch
Sent: Thursday, August 19, 2010 7:52 AM
To: EPICS
Subject: CA gateway dies without error message

Hi all,

Since upgrading the EPICS base version of the caGateways to 3.14.11,
the
gateway dies from time to time for no obvious reason. It does not write
any log message when it dies (as it would do in a clean shutdown, in
case of a signal or when any strange CA event happens.) It also does
not
write anything to stderr. I cannot remember that I have seen this type
of fault with 3.13.8.

Any idea?

Dirk




Replies:
RE: CA gateway dies without error message Jeff Hill
References:
CA gateway dies without error message Dirk Zimoch
RE: CA gateway dies without error message Jeff Hill

Navigate by Date:
Prev: msi again Benjamin Franksen
Next: Re: hex stream processing with StreamDevice? Dirk Zimoch
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: CA gateway dies without error message Jeff Hill
Next: RE: CA gateway dies without error message Jeff Hill
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 02 Sep 2010 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·