EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Deadlock in epicsQT
From: Zenon Szalata <[email protected]>
To: Andrew Rhyder <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>
Date: Fri, 22 Aug 2014 22:08:54 -0700
I am sending this note again since the screen shot attached was too long to be normally posted.  This time I am attaching a gdb trace back file.  Small correction to the original text, "It always involves ca_clear_channel routine" is not really true.
Here is the original text:
I am sending this to a wider audience hoping to improve chances for a 
useful suggestion on how to proceed solving this problem.
Here is a description of the problem:
My epicsQT application at times fails to come up properly.  Each time it 
fails, attaching a debugger reveals that it is a waiting on a mutex, 
which it never gets, so it hangs forever.  An example of a traceback is 
attached.  It does not get stuck in the same place each time it 
happens.  It always involves ca_clear_channel routine.

The affected application is fairly large.  There are more than 500 EPICS 
aware widgets.  The application gets stuck usually after about 250 
channel access connections are made.  Sometimes it happens much sooner 
and sometimes it makes it all the way.  The application is coded such 
that in the main window there are number of nested tabbed widgets.

Since most of the time the problem seems to occur after a substantial 
number of CA connections are made, I decided to create a variant of this 
application.  The new variant consists of a main window with a number of 
push buttons each designed to launch a dialog.  The class object is 
created when the dialog is launched. Each of these dialog windows 
contains the widgets from each page of the tabbed widgets of the 
original application.  This way, each dialog contains  a modest number 
of CA aware widgets.  I have implemented signals and slots such that 
when a dialog is dismissed, a signal is emitted which the main window 
intercepts and destroys the dialog objects.  The end result is that this 
application can be always started and hang only when number of these 
dialog windows are popped up or put it differently, when there are a 
large number of active CA connection in this application and a new 
dialog is popped up.

I have not spent much time reading the epicsQT code so I don't really 
know why it is doing what it is doing.  I can't tell if the fault is in 
the epicsQT implementation or is it perhaps some resource depletion 
issue.  I hope for some suggestion as to how to proceed diagnosing this.

EPICS version is 3.14.12.4
epicsQT is 2.8.1
qt is 4.8.5
The computer is RHEL6 64 bits with quad core Xeon W3550, 6GB memory.

Thank you in advance,
Zen

(gdb) bt
#0  0x0000003fd1c0e264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000003fd1c09523 in _L_lock_892 () from /lib64/libpthread.so.0
#2  0x0000003fd1c09407 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007fb529267086 in mutexLock (pmutex=0x1438540)
    at ../../../src/libCom/osi/os/posix/osdMutex.c:44
#4  epicsMutexOsdLock (pmutex=0x1438540)
    at ../../../src/libCom/osi/os/posix/osdMutex.c:116
#5  0x00007fb5292608ac in epicsMutex::lock (this=<value optimized out>)
    at ../../../src/libCom/osi/epicsMutex.cpp:236
#6  0x00007fb5294cbdd0 in epicsGuard (pMon=0x7fb510003570)
    at ../../../include/epicsGuard.h:71
#7  CallbackGuard (pMon=0x7fb510003570) at ../cacIO.h:159
#8  ca_clear_subscription (pMon=0x7fb510003570) at ../ca_client_context.cpp:781
#9  0x00007fb52acdbb38 in caconnection::CaConnection::removeChannel (this=
    0x1a69a20) at api/src/CaConnection.cpp:263
#10 0x00007fb52acd86db in CaObjectPrivate::removeChannel (this=0x1a69960)
    at api/src/CaObject.cpp:186
#11 0x00007fb52ace136e in qcaobject::QCaObject::~QCaObject (this=0x1a69750, 
    __in_chrg=<value optimized out>) at data/src/QCaObject.cpp:169
#12 0x00007fb52ae2c842 in QEString::~QEString (this=0x1a69750, 
    __in_chrg=<value optimized out>) at data/include/QEString.h:36
#13 0x00007fb52ae2c880 in QEString::~QEString (this=0x1a69750, 
    __in_chrg=<value optimized out>) at data/include/QEString.h:36
#14 0x00007fb52acf6a5b in QEWidget::deleteQcaItem (this=0x1a0db48, 
    variableIndex=0, disconnect=false) at widgets/src/QEWidget.cpp:286
#15 0x00007fb52acf6851 in QEWidget::createConnection (this=0x1a0db48, 
    variableIndex=0) at widgets/src/QEWidget.cpp:201
#16 0x00007fb52ad85cd9 in QELabel::establishConnection (this=0x1a0db20, 
    variableIndex=0) at widgets/QELabel/QELabel.cpp:98
#17 0x00007fb52acf67a5 in QEWidget::activate (this=0x1a0db48)
    at widgets/src/QEWidget.cpp:168
#18 0x00007fb52ad3d7a4 in QEForm::readUiFile (this=0x1eed390)
    at widgets/QEForm/QEForm.cpp:395
#19 0x00007fb52ad3c938 in QEForm::reloadLater (this=0x1eed390)
    at widgets/QEForm/QEForm.cpp:186
#20 0x00007fb52ae42ca5 in QEForm::qt_static_metacall (_o=0x1eed390, _c=
    QMetaObject::InvokeMetaMethod, _id=3, _a=0x1ef31a0) at moc_QEForm.cpp:79
#21 0x00007fb5298a6a6e in QObject::event(QEvent*) ()
   from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#22 0x00007fb529e33c3c in QWidget::event(QEvent*) ()
   from /opt/qt/qt-4.8.5/lib/libQtGui.so.4
#23 0x00007fb529de8e5c in QApplicationPrivate::notify_helper(QObject*, QEvent*)
    () from /opt/qt/qt-4.8.5/lib/libQtGui.so.4
#24 0x00007fb529def0f1 in QApplication::notify(QObject*, QEvent*) ()
   from /opt/qt/qt-4.8.5/lib/libQtGui.so.4
#25 0x00007fb529895b0c in QCoreApplication::notifyInternal(QObject*, QEvent*)
    () from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#26 0x00007fb5298999a3 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#27 0x00007fb5298c1ca3 in postEventSourceDispatch(_GSource*, int (*)(void*), void*) () from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#28 0x0000003fd2c3feb2 in g_main_context_dispatch ()
   from /lib64/libglib-2.0.so.0
#29 0x0000003fd2c43d68 in ?? () from /lib64/libglib-2.0.so.0
#30 0x0000003fd2c43f1c in g_main_context_iteration ()
---Type <return> to continue, or q <return> to quit---
   from /lib64/libglib-2.0.so.0
#31 0x00007fb5298c17e3 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#32 0x00007fb529e8921e in QGuiEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/qt/qt-4.8.5/lib/libQtGui.so.4
#33 0x00007fb5298949a2 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#34 0x00007fb529894d04 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#35 0x00007fb529899d49 in QCoreApplication::exec() ()
   from /opt/qt/qt-4.8.5/lib/libQtCore.so.4
#36 0x000000000040662a in main ()
(gdb) 


Navigate by Date:
Prev: SNL syntax highlight for text editors Mazanec Tomáš
Next: Re: SNL syntax highlight for text editors Florian Feldbauer
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: SNL syntax highlight for text editors Mazanec Tomáš
Next: Syntax highlighting for db or protocol files Mazanec Tomáš
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 17 Dec 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·