Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: crash in access security when other ioc reboots
From: Dirk Zimoch <dirk.zimoch@psi.ch>
To: EPICS <tech-talk@aps.anl.gov>
Date: Tue, 13 Jun 2017 12:04:01 +0200
Hi all,

Maybe someone can help me diagnose this strange problem:

We have one IOC with access security rules using a PV from another IOC. When that other IOC is restarted, this IOC crashes.

Has anyone seen such behavior before?
I am using EPICS base 3.14.12.4.

Here is a stack trace:
#0  0x0fd1e798 in ellDelete (pList=0xb5113d90, pNode=0x10c7e928)
    at ../../../src/libCom/ellLib/ellLib.c:87
#1  0x0ff01f40 in casAccessRightsCB (ascpvt=<optimized out>,
    type=<optimized out>) at ../camessage.c:1111
#2  0x0feadb0c in asComputePvt (asClientPvt=0x10cb02d0)
    at ../asLibRoutines.c:1014
#3  0x0feaddbc in asComputePvt (asClientPvt=0x10cb02d0)
    at ../asLibRoutines.c:959
#4  asComputeAsgPvt (pasg=<optimized out>) at ../asLibRoutines.c:940
#5  0x0feae880 in asComputeAsgPvt (pasg=0x0) at ../asLibRoutines.c:918
#6  asComputeAsg (pasg=0x0) at ../asLibRoutines.c:455
#7  0x0feacd24 in connectCallback (
arg=<error reading variable: value has been optimized out>) at ../asCa.c:99
#8  0x0fdb488c in oldChannelNotify::disconnectNotify (this=0x10a11800,
    guard=...) at ../oldChannelNotify.cpp:112
#9  0x0fda2908 in nciu::unresponsiveCircuitNotify (this=0x10a2b348,
    cbGuard=..., guard=...) at ../nciu.cpp:171
#10 0x0fdac83c in tcpiiu::disconnectAllChannels (this=0x10a51370, cbGuard=...,
    guard=..., discIIU=...) at ../tcpiiu.cpp:1834
#11 0x0fd95a98 in cac::destroyIIU (this=0x10a19808, iiu=...) at ../cac.cpp:1227
#12 0x0fdace54 in tcpSendThread::run (this=0x10a51458) at ../tcpiiu.cpp:229
#13 0x0fd30750 in epicsThreadCallEntryPoint (pPvt=0x10a5145c)
    at ../../../src/libCom/osi/epicsThread.cpp:85
#14 0x0fd37868 in start_routine (arg=0x10412000)
#15 0xb7b9382c in start_thread (arg=0xb04f8480) at pthread_create.c:306
#16 0xb7c9e0e4 in clone () from /lib/libc.so.6

In ellDelete, pNode looks like this:
(gdb) print *pNode
$12 = {next = 0x10c7e838, previous = 0x0}

This crashes when line 87 tries this:
        pNode->previous->next = pNode->next;

I wonder how *pNode could get into this (inconsistent) state?

In casAccessRightsCB, pciu looks like this:
(gdb) print *pciu
$17 = {node = {next = 0x10c7e838, previous = 0x0}, eventq = {node = {
      next = 0x0, previous = 0x0}, count = 0}, client = 0xb5113d50,
  pPutNotify = 0x0, cid = 1, sid = 247215, time_at_creation = {
secPastEpoch = 866178863, nsec = 248388556}, addr = {precord = 0x10777238,
    pfield = 0x107773a0, pfldDes = 0x10035270, no_elements = 1,
    field_type = 9, field_size = 2, special = 0, dbr_field_type = 3},
  asClientPVT = 0x10cb02d0, state = rsrvCS_inService}

For example:
(gdb) print pciu->addr.precord->name
$18 = "SINSB01-RILK-PRE:POWERON", '\000' <repeats 36 times>

This is one of the records using the ASG rule with the external PV.

However pciu->node.previous is NULL which causes the crash.

Also the pciu->node.next points to a structure for a record using thus ASG rule:

print ((struct channel_in_use*)pciu->node.next)->addr.precord->name
$19 = "SINSB01-RILK:MPS-RESET", '\000' <repeats 38 times>

But again here, previous is NULL:
print *pciu->node.next
$20 = {next = 0x10c7e888, previous = 0x0}


Any ideas?

Dirk



Replies:
Re: crash in access security when other ioc reboots Michael Davidsaver

Navigate by Date:
Prev: Re: IOC Severity Report Ralph Lange
Next: Re: crash in access security when other ioc reboots Michael Davidsaver
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Re: IOC Severity Report Matt Rippa
Next: Re: crash in access security when other ioc reboots Michael Davidsaver
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 13 Jun 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·