Hi Phil,
I would like to understand the situation in which this could be happening.
It seems like you must be doing the following:
1.
Allocating an NDArray with pData pointing to a
memory buffer that you have allocated and manage myself
2.
Returning that array to the pool with NDArrayPool->release()
3.
Allocating another array from the pool with pData null, but with a size that is larger than the one you allocated in step 1.
Is this the case? If so, then I do see the problem. It finds a free array in the pool, then finds that its size it not large enough, so it calls free() to free that memory and allocated new memory. Because
you used some other mechanism to allocate your memory it crashes.
This is a scenario I had not envisioned. We could handle this by setting a flag that says that if the array was allocated with pData not null then do not call free(). In this case it would need to keep searching
for another existing NDArray that was large enough, and if none is found then allocated a new one. This has 2 disadvantages:
-
It complicates the allocation process
-
It potentially leads to significant memory leaks. The NDArray that you allocated may never be able to be used again because there are no requests that are small enough.
The simplest solution is just to document the restriction that if you pass a non-null pData it must have been allocated with some mechanism that is compatible with free().
Sinisa Veseli
[email protected] has a patch for NDArray that allows them to subclass it, because they want to store additional information in the NDArray. Perhaps it would help with this problem. I think not though, because this is really a problem with NDArrayPool
and not NDArray itself.
Mark
In my areaDetector plugin I'm calling this method in one of two ways, depending on a PV value:
-
With pData pointing to a memory buffer that I've allocated and manage myself;
-
With pData null
All seems to work well when operated continuously in either mode, but I see a crash in RtlFreeHeap when I process at least one frame in the first mode, and then switch to the second. This does not happen if I perform the same sequence of steps but instead
of passing the pData to my buffer, I pass null and then copy from my buffer to the output - but this is a large buffer so I'd like to avoid this.
I haven't found anything in the documentation that says as much but I had assumed that, if I pass my own buffer to alloc, NDArrayPool would regard me as its owner and would never try to deallocate it. (I would also expect it to exclude my buffer's size
from its allocation limit calculations, but that's a secondary concern). However, that appears not to be the case: NDArrayPool::alloc is deciding to free a now-unused NDArray that's pointing to my buffer and so it's calling free(freeArray->pData); that's
failing because the memory was allocated in a manner incompatible with free. NDArray seems to contain nothing that indicates who allocated the buffer, so NDArrayPool cannot reasonably alter its behaviour accordingly.
This behaviour doesn't seem right to me (on general principles) - but maybe I've misunderstood how I'm supposed to use this API?
Thanks,
Phil
--
Pixel Analytics is a limited company registered in England. Company number:
7747526; Registered office: 93A New Road, Haslingfield, Cambridge CB23 1LP