Experimental Physics and
Industrial Control System

"Denison, PN \(Peter\)" <[email protected]> · Mon, 17 Sep 2007 22:34:22 +0100

I'm trying to debug a deadlock on Linux under R3.14.8.2, on a cut down
IOC with only asyn, motor and autosave installed. Autosave is only
required to cause the motor to execute the particular code that causes
the deadlock.

motor 6-2-2 (with minor local mods)
asyn 4-8
epics base R3.14.8.2
RedHat Enterprise Linux 4 (glibc-2.3.4-2.25, gcc 3.4.6-3)

What appears to be happening is that one thread is waiting on an
epicsEvent, whilst another thread is trying to signal that event, but
getting stuck on the event's internal lock. The pthread man pages imply
that this shouldn't happen.

I don't think it's a classical priority inversion, as there is no
resource starvation - all threads are waiting or sleeping.

There may well be a flaw in my motor code in devMotorAsyn.c, that's
exposing this, but it still shouldn't happen.

It doesn't happen under vxWorks, only Linux (which is entirely
reasonable, as the event code being called is OS-specific).

Any clues?

===== Detail =====

After interrupting the deadlock, there are 12 threads running:
10 standard ones, sleeping, or waiting on perfectly reasonable things

and the 2 interesting ones:
asynCallback() calling epicsEventSignal(pPvt->initEvent), still trying
to lock &pevent->mutex:
#10 in epicsEventSignal() at osdEvent.c:54
#9 in pthread_mutex_lock() from libc.so.6
#8 in pthread_mutex_lock() from libc.so.6
#7 in ??()
#6 in ??()
#5 in epicsTime::epicsTime() from libCom.so
#4 in ??() from libc.so.6
#3 in ??()
#2 in _L_mutex_lock_215() from libpthread.so.0
#1 in __lll_mutex_lock_wait() from libpthread.so.0
#0 in _dl_sysinfo_int80 from ld-linux.so.2

and main iocInit thread calling motor initialisation:
#6 in iocInit() at iocInit.c:364
#5 in init_record() at motorRecord.cc:482
#4 in init_record() at devMotorAsyn.c:138
       epicsEventMustWait(initEvent);
#3 in epicsEventWait() at osdEvent.c:75
#2 in pthread_cond_wait@@GLIBC_2.3.2() from libc.so.6
#1 in pthread_cond_wait@@GLIBC_2.3.2() from libpthread.so.0
#0 in _dl_sysinfo_int80() from ld-linux.so.2

To reproduce it, create an ioc with 2 motor records, DTYP=asynMotor,
OUT=@asyn(name,0) / (name,1), with the following 2 lines in st.cmd:
motorSimCreate(0, 0, -50000, 50000, 0, 1, 2)
drvAsynMotorConfigure("name", "motorSim", 0, 2)
and motorSupport.dbd, motorSimSupport.dbd, asyn.dbd, asSupport.dbd in
the DBD
and motorSimSupport, motor, asyn, autosave in the LIBS
Arrange for the DVAL of the 2 motor records to be restored by autosave
(by creating a .sav file, and using set_pass0_restoreFile()

-- 
Peter Denison, Senior Software Engineer, Diamond Light Source Ltd.
Tel: +44 1235 778511
(apologies in advance for the lines below. Some bits are a legal
requirement and I have no control over them)

<DIV><FONT size="1" color="gray">This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
</FONT></DIV> 

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System