g+
g+ Communities
Argonne National Laboratory

Experimental Physics and
Industrial Control System

2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014  Index 2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014 
<== Date ==> <== Thread ==>

Subject: memory management
From: Jeff Hill <johill@lanl.gov>
To: Marty Kraimer <mrk@aps.anl.gov>, Andrew Johnson <anj@aps.anl.gov>
Cc: 'Benjamin Franksen' <benjamin.franksen@bessy.de>, 'Ralph Lange' <Ralph.Lange@bessy.de>, 'Eric Norum' <norume@aps.anl.gov>, 'Bob Dalesio' <bdalesio1@comcast.net>, Ken Evans <evans@aps.anl.gov>, 'Benjamin Franksen' <benjamin.franksen@bessy.de>, 'Ken Evans' <evans@aps.anl.gov>, 'Matej Sekoranja' <matej.sekoranja@ijs.si>, "Kasemir, Kay" <kasemirk@ornl.gov>
Date: Thu, 24 Feb 2005 10:57:07 -0700
All,

Perhaps it's worthwhile to review one of the key design goals for
data access - interfaces permitting non-contiguous array storage
and improved memory management.

The executive summary is that, with the C RTL, continuously
allocating random sized blocks has a very bad impact on memory
fragmentation, and therefore CPU overhead. This leads to
reservations about temporary storage of arrays in non-contiguous
fixed sized blocks, and therefore free list based memory
management appears to be a desirable goal.

Free list based memory management for CA protocol buffers is
appropriate from a memory management perspective for the reasons
stated above. We also like the idea of no longer needing to
configure EPICS_CA_MAX_ARRAY_BYTES.

However, consider that dbPut / dbGet as written today both
require contiguous array arguments. Their current interface
prevents use of fixed sized non-contiguous blocks for storage of
arrays in protocol buffers, and therefore the current design of
this interface probably needs to be carefully considered when
designing upgrades.

Free list based buffer memory management is also good for the
event queue because we can store a copy of the array on the event
queue when the event is posted. Caving in to overhead and
fragmentation concerns, my original design (and current practice)
is to store only knowledge that the array has changed on the
event queue. The server is currently sending array subscription
updates using whatever might be in the array when the event is
pulled from the queue by the CA server, and not what was in the
array when the event was posted by the higher priority record
processing thread. The overhead to allocate space for the array
when the event is posted will be quite reasonable if a free list
is used, and of course if there are multiple clients subscribed
to the same field we can have multiple references to a single
array storage from multiple event queues if reference counting is
employed. 

Finally, please indulge a brief detour related to reference
counted buffer sharing. This technique can reduce copying for
large arrays. With this approach a reference count is maintained
keeping track of the number of references (pointers) to a buffer.
When the reference count goes to zero the buffer is automatically
returned to its free list. Of course this level of sophistication
would require some type of smart pointer entity ( dare I use the
word class or object :) that reference counts a non-contiguous
array and also automatically frees storage for the array when the
reference count decrements to zero.

Consider a scenario where the event queue would not create a copy
of the array when an event is posted, but instead increment the
reference count for what is stored in the database. When the
database record processes again it would notice that its array
storage reference count is greater than one, decrement the
reference count for its array storage, allocate new array storage
from a free list, and copy the input link value into the new
storage. Alternatively, if the reference count for the database's
array storage is one, then the database might need only to copy
the new value into its preexisting storage, and not allocate new
storage. This approach might be used to avoid overhead in
situations where the CA server processes the event queue entry
prior to when the database record processes again. 

Similar types of benefits (reduced copying) might occur with the
dbGet and dbPut interfaces if reference counted buffer sharing
were employed.

Thanks for your patience with a long message. I am not proposing
a design here - only attempting to engage your attention to some
issues that probably need to be considered, and others that might
be considered early if they will be used.

Jeff
__________________________________________________________
Jeffrey O. Hill               Mail         JOHill@lanl.gov
LANL MS H820                  Voice        505 665 1831
Los Alamos NM 87545 USA       Fax          505 665 5107





Navigate by Date:
Prev: Re: EPICS base V4 Benjamin Franksen
Next: RE: memory management Jeff Hill
Index: 2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014 
Navigate by Thread:
Prev: Re: [Fwd: RE: EPICS base V4: iocCore database] Benjamin Franksen
Next: RE: memory management Jeff Hill
Index: 2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014 
ANJ, 02 Feb 2012 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICSv4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·