Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: Re: memory leak after unloading of ca.lib
From: Michael Davidsaver <mdavidsaver@gmail.com>
To: Carsten Winkler <carsten.winkler@helmholtz-berlin.de>
Cc: "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Fri, 24 Feb 2017 10:56:04 -0500
On 02/24/2017 06:26 AM, Carsten Winkler wrote:
> Hi there,
> 
> thank you for all response. It can be summarized as follows:
> 
> #1 libCom has the errlog thread that is never stopped.

This was one example, there are others (run epicsThreadShowAll() to
see).  eg. the worker thread behind the async. DNS API
(ipAddrToAsciiEngine).  This one used to be stopped, but no isn't to
avoid blocking short lived clients like caget.  There may also be
thread(s) related to clock synchronization.

> #2 libCa registers a function pointer using epicsAtExit and if it will
> fail if if libca is unloaded.
> #3 There are also a number of one-time allocations which aren't cleaned up.
> #4 It has something to do with the ca repeater process
> #5 It has something to do with LabVIEW and it's memory management
> #6 Ca context destroy doesn't clean up components in libCom and other
> dependent libraries
> #7 Question whether libCom will be unloaded
> #8 Cleaning up with epicsExit()
> 
> 
> My questions/answers to this points:
> 
> #1 Where/when exactly this thread is started? How can I terminate it?

see src/libCom/error/errlog.c.  The present API doesn't provide a way.

> #2 When will the function epicsAtExit called in my demo?

I'm not sure.  RTEMS, WIN32, and all posix targets make use of atexit()
from libc to hook in to process termination.  The posix targets use this
to call epicsExitCallAtExits().  It isn't immediately clear to me if
WIN32 does this as well.

>     Do you think, this function pointer leaves bad memory sections in my
> case?

Yes.

> #3 That's what I'm looking for. Do I have any chance to resolve it?

It is fixable in principle.  Some progress is made in Base >3.14 to
partially shutdown and restart process database facilities to support
unit tests.  This was a lot of work.

> #4 No, the caRepater is not involved. When I run my test app there is no
> caRepeater process running.
> #5 No, LabVIEW was the trigger only to investigate the issue. But every
> program with plugin interface will have the same problem. The most
> simple example is my test case.

I haven't come across any plugin interfaces which try to unload
arbitrary libraries.  Specifically written plugin libs, yes.  Just not
their dependencies.

> #6 How can I do the necessary clean up?
> #7 In Windows I see that Com.dll won't be unloaded. I guess it's the
> same in Linux. How can I detect loaded libs at runtime?

Not a windows expert, but I remember that the dllmain() callback is
involved in dll (un)loading.

>      Will unloading of libCom solve the issue?

Unless all threads are stopped, this will trigger an immediate crash.
If all threads are stopped, then it will simply leak memory.

> #8 The function epicsExit() ends with exit(status); and will terminate
> calling program too. That is what I try to prevent.
> 
> Phew, it seems to be quite complicated. Hope we find a solution together.

There will be even more fun if/when you try to re-load libca.

If I were in your place, I'd focus on trying to avoid unloading libca.
If libCom isn't being unloaded, then you can make a stub/no-op library
which has libca and libCom as dependencies.  Then only the stub library
should be unloaded.


Replies:
Re: memory leak after unloading of ca.lib Carsten Winkler (HZB)
References:
memory leak after unloading of ca.lib Carsten Winkler
Re: memory leak after unloading of ca.lib Michael Davidsaver
Re: memory leak after unloading of ca.lib Ben Franksen
RE: memory leak after unloading of ca.lib Hill, Jeff
Re: memory leak after unloading of ca.lib Michael Davidsaver
RE: memory leak after unloading of ca.lib Hill, Jeff
Re: memory leak after unloading of ca.lib Michael Davidsaver
Re: memory leak after unloading of ca.lib Carsten Winkler

Navigate by Date:
Prev: Re: memory leak after unloading of ca.lib Carsten Winkler
Next: streamDevice protocol - accept two different valid responses James.OHea
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Re: memory leak after unloading of ca.lib Carsten Winkler
Next: Re: memory leak after unloading of ca.lib Carsten Winkler (HZB)
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 24 Feb 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·