Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: RE: Channel Access client sees only partial update.
From: Mark Rivers <rivers@cars.uchicago.edu>
To: 'Michael Huth' <michael.huth@byte-physics.de>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Tue, 6 Jun 2017 19:38:57 +0000
Hi Michael,

What version of asyn are you using?

Can you send a longer sequence of blocks received with and without the info tag?

> A rough guess of me is that the copy to the ring buffer is late and the
> source was already updated with the next block?

The copy to the ring buffer in done in your driver's thread that is calling doCallbacksInt8Array(blockPtr , block_size, Data, 0).  There is no way it can be "late".

The only way I can see for the data to be overwritten is if your driver has another thread which is modifying the data in blockPtr while doCallbacksInt8Array() is executing.  If so you need to use a mutex to make sure you don't change the data in blockPtr before doCallbacksInt8Array has returned.

In areaDetector normally the data is copied into an NDArray object that is allocated from the NDArrayPool.  The driver calls doCallbacksGenericPointer() to pass the NDArray to the plugins.  The NDPluginStdArrays plugin is what would actually call doCallbacksInt8Array.  It then decrements the reference count on the NDArray, returning it to the NDArrayPool if this was the last reference.  This prevents overwriting buffers if you have another thread getting the data from the device because it would always allocate a new NDArray.

> Is there a way to block execution in the IOC to be sure that the data is in the Channel Access output buffer?

You don't need to wait for it to be in the Channel Access buffer, that is the point of the ring buffers.  The waveform record device support can buffer multiple arrays in the ring buffer.  You just need to make sure that your driver does not modify the data pointed to by blockPtr until doCallbacksInt8Array() has returned.

Mark



-----Original Message-----
From: Michael Huth [mailto:michael.huth@byte-physics.de] 
Sent: Tuesday, June 06, 2017 2:14 PM
To: Mark Rivers; tech-talk@aps.anl.gov
Subject: Re: Channel Access client sees only partial update.

Hello Mark,

with the line added to the database file the behavior is not much different,
just that I get now sometimes really multiple blocks per IOC task loop,
however with block IDs like this:

Block 1037
Block 1038
Block 1041
Block 1041

where e.g. a block with the 1041 data is sent twice. The ID number is
part of the blocks data.
A rough guess of me is that the copy to the ring buffer is late and the
source was already updated with the next block?
Is there a way to block execution in the IOC to be sure that the data is
in the Channel Access output buffer?

The HDF5 swmr is actually already implemented in the CA client. It is
not planned to use HDF5 writing as part of the IOC driver.

Best regards,
Michael

On 06.06.2017 20:28, Mark Rivers wrote:
> Hi Michael,
>
> By default the devAsynInt8Array callback does not buffer the array it receives, it copies it directly into the waveform record.  If your driver callbacks are happening faster than the record processing and Channel Access transfer to the client then what you are seeing is expected.
>
> devAsynInt8Array can optionally use a ring-buffer to buffer the arrays passed to the callback function.  This should eliminate the problem you are seeing.  The ring buffer is disabled by default, but you can enable it by adding this line to your waveform record database file:
>
> info(asyn:FIFO, "20")
>
> where 20 is the maximum number of blocks you want to be able to buffer.
>
> This is documented here (search for "Buffering of Driver Callbacks"):
> http://www.aps.anl.gov/epics/modules/soft/asyn/R4-31/asynDriver.html
>
> How large is the data set you are trying to transfer?  Are you breaking it into blocks because of its size, or is your source actually delivering it in sub-blocks?
>
> Mark
>
>
> -----Original Message-----
> From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Michael Huth
> Sent: Tuesday, June 06, 2017 1:15 PM
> To: tech-talk@aps.anl.gov
> Subject: Channel Access client sees only partial update.
>
> Hello,
>
> I'am quite new to Epics and ran into an issue where I could use a hint
> how to solve it:
>
> I have an IOC driver based on the area detector driver that should be
> extended for the transfer of a large data set from another source.
>
> So I setup a Record for it like this:
> record(waveform, "$(P)$(R)Data")
> {
>     field(PINI, "0")
>     field(DTYP, "asynInt8ArrayIn")
>     field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))DATA")
>     field(FTVL, "CHAR")
>     field(NELM, "$(SIZE_DATA_BLOCK)")
>     field(SCAN, "I/O Intr")
> }
>
> In the main thread of the IOC I split the large data set into smaller
> blocks and have a loop that calls
>
> doCallbacksInt8Array(blockPtr , block_size, Data, 0);
>
> for each block to transfer the blocks to the data base.
>
> In a second process a channel access client is subscribed with
> ca_create_subscription to 'DATA'.
>
> The issue is that the client receives only the last block from each loop
> of the IOC driver thread.
>
> I guess I have to wait somehow until the database processing has
> finished after each block?!?
>
> Best regards,
> Michael Huth
>

-- 
() byte physics
    Dr. Michael Huth
    Software-Developer    

Company CEO: Thomas Braun
Address:     Schwarzastraße 9, 12055 Berlin, Germany
Web:         www.byte-physics.de
Telephone:   +49 (0)30/52 68 23 22
Fax:         +49 (0)30/52 68 43 26
USt-IdNr:    DE276717536


Replies:
Re: Channel Access client sees only partial update. Michael Huth
References:
Channel Access client sees only partial update. Michael Huth
RE: Channel Access client sees only partial update. Mark Rivers
Re: Channel Access client sees only partial update. Michael Huth

Navigate by Date:
Prev: Re: Channel Access client sees only partial update. Michael Huth
Next: Re: Channel Access client sees only partial update. Michael Huth
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Re: Channel Access client sees only partial update. Michael Huth
Next: Re: Channel Access client sees only partial update. Michael Huth
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 06 Jun 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·