Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: Re: Channel Access client sees only partial update.
From: Michael Huth <michael.huth@byte-physics.de>
To: Mark Rivers <rivers@cars.uchicago.edu>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Tue, 6 Jun 2017 20:46:36 +0200
Hello Mark,

thanks for the hint.
The instrument streams via USB3 (in theory up to 400 MB/s) into it's own
buffer. So the data set is of variable size and could grow fast.
Probably ~100 MB minimum and may be up to 10 GB+.
I wanted to split it into blocks to be able to find an optimal size for
fastest transfer to the channel access client. It might also prevent
long pauses from large memory allocations.

Best regards,
Michael

On 06.06.2017 20:28, Mark Rivers wrote:
> Hi Michael,
>
> By default the devAsynInt8Array callback does not buffer the array it receives, it copies it directly into the waveform record.  If your driver callbacks are happening faster than the record processing and Channel Access transfer to the client then what you are seeing is expected.
>
> devAsynInt8Array can optionally use a ring-buffer to buffer the arrays passed to the callback function.  This should eliminate the problem you are seeing.  The ring buffer is disabled by default, but you can enable it by adding this line to your waveform record database file:
>
> info(asyn:FIFO, "20")
>
> where 20 is the maximum number of blocks you want to be able to buffer.
>
> This is documented here (search for "Buffering of Driver Callbacks"):
> http://www.aps.anl.gov/epics/modules/soft/asyn/R4-31/asynDriver.html
>
> How large is the data set you are trying to transfer?  Are you breaking it into blocks because of its size, or is your source actually delivering it in sub-blocks?
>
> Mark
>
>
> -----Original Message-----
> From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Michael Huth
> Sent: Tuesday, June 06, 2017 1:15 PM
> To: tech-talk@aps.anl.gov
> Subject: Channel Access client sees only partial update.
>
> Hello,
>
> I'am quite new to Epics and ran into an issue where I could use a hint
> how to solve it:
>
> I have an IOC driver based on the area detector driver that should be
> extended for the transfer of a large data set from another source.
>
> So I setup a Record for it like this:
> record(waveform, "$(P)$(R)Data")
> {
>     field(PINI, "0")
>     field(DTYP, "asynInt8ArrayIn")
>     field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))DATA")
>     field(FTVL, "CHAR")
>     field(NELM, "$(SIZE_DATA_BLOCK)")
>     field(SCAN, "I/O Intr")
> }
>
> In the main thread of the IOC I split the large data set into smaller
> blocks and have a loop that calls
>
> doCallbacksInt8Array(blockPtr , block_size, Data, 0);
>
> for each block to transfer the blocks to the data base.
>
> In a second process a channel access client is subscribed with
> ca_create_subscription to 'DATA'.
>
> The issue is that the client receives only the last block from each loop
> of the IOC driver thread.
>
> I guess I have to wait somehow until the database processing has
> finished after each block?!?
>
> Best regards,
> Michael Huth
>

-- 
() byte physics
    Dr. Michael Huth
    Software-Developer    

Company CEO: Thomas Braun
Address:     Schwarzastraße 9, 12055 Berlin, Germany
Web:         www.byte-physics.de
Telephone:   +49 (0)30/52 68 23 22
Fax:         +49 (0)30/52 68 43 26
USt-IdNr:    DE276717536


Replies:
RE: Channel Access client sees only partial update. Mark Rivers
References:
Channel Access client sees only partial update. Michael Huth
RE: Channel Access client sees only partial update. Mark Rivers

Navigate by Date:
Prev: RE: Channel Access client sees only partial update. Mark Rivers
Next: RE: Channel Access client sees only partial update. Mark Rivers
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: RE: Channel Access client sees only partial update. Mark Rivers
Next: RE: Channel Access client sees only partial update. Mark Rivers
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 06 Jun 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·