EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  <20152016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  <20152016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: areaDetector plugin: very large output array
From: Phil Atkin <[email protected]>
To: Mark Rivers <[email protected]>, "[email protected]" <[email protected]>
Date: Wed, 16 Dec 2015 19:41:52 +0000
Just to re-iterate: the contents of the large array must be preserved over many cycles.  There can only be a single large array in use; the processing produces a cumulative result in it.  (Imagine, if you will, a plugin that extrapolates each input image to a larger array, then sums the result; this is structurally equivalent to what I need although the details are different).

Does this change the answer?

Phil

On 16/12/2015 17:57, Mark Rivers wrote:
I would first see if the normal mechanism of allocating an array from the pool will work.  I suspect it will work fine.

That mechanism only needs to allocate more than 2 arrays at a time if a downstream plugin is still busy processing the previous array when your plugin does callbacks on the next array.  If that does not happen, and all downstream plugins are done by the time you do your next callback then 2 pool arrays should be enough.  That's only 2 GB of RAM which any modern machine should not have trouble with.

You can also set all downstream plugins to have CallbacksBlock=Yes, at which point there is no queueing of arrays at all, all the plugins are run in the thread for your plugin, and a single array would be enough.

Mark

________________________________
From: [email protected] [[email protected]] on behalf of Phil Atkin [[email protected]]
Sent: Wednesday, December 16, 2015 10:39 AM
To: [email protected]
Subject: areaDetector plugin: very large output array

Hi folks,

My plugin needs to build and output a very large image - 1GB would not be surprising.  The contents of the image are gradually accumulated as many images are produced by the detector driver.  I'm not sure how I can make this 'fit' the NDArray/NDArrayPool architecture.  If I make the accumulator image a private area of memory then I could copy to an output NDArray when required - but that requires even more memory.

My ideal approach would be for my plugin to allocate the accumulator array (a 'worker' class currently encapsulates this), and only when the output is required would a pointer to this memory be used to 'publish' the result through an NDArray.

If I created a private NDArray and set its pData pointer to my accumulator image, could I then pass this to doCallbacksGenericPointer instead of the usual this->pArrays[0]?  If so, that call would presumably be inexpensive unless/until a client subscribed to the output port?

Thanks,

Phil
--
[cid:[email protected]]Pixel Analytics is a limited company registered in England. Company number: 7747526; Registered office: 93A New Road, Haslingfield, Cambridge CB23 1LP


--
Pixel Analytics is a limited company registered in England. Company number: 7747526; Registered office: 93A New Road, Haslingfield, Cambridge CB23 1LP

Replies:
Re: areaDetector plugin: very large output array Jason Matthias Abernathy
References:
areaDetector plugin: very large output array Phil Atkin
RE: areaDetector plugin: very large output array Mark Rivers

Navigate by Date:
Prev: Re: epics docker container Carlos Pascual
Next: Re: areaDetector plugin: very large output array Jason Matthias Abernathy
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  <20152016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: areaDetector plugin: very large output array Mark Rivers
Next: Re: areaDetector plugin: very large output array Jason Matthias Abernathy
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  <20152016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 16 Dec 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·