Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017 
<== Date ==> <== Thread ==>

Subject: Re: labCA and EPICS_CA_MAX_ARRAY_BYTES
From: Ralph Lange <Ralph.Lange@bessy.de>
To: EPICS Tech Talk <tech-talk@aps.anl.gov>
Date: Wed, 26 May 2010 12:27:10 -0400
Hi Matt,

nope, it's the other way round:

Neither the IOC nor the on-the-wire protocol limit the size of arrays to be transferred.

CA server and client libraries internally use a carefully crafted mechanism of fixed-size buffers on free lists to avoid cluttering up the memory. This is especially important for servers, since IOCs may have limited memory and high availability requirements.
The largest fix-size buffers in that mechanism are by default 16384 bytes, which is more a historical than a technical value.
At some point, EPICS_CA_MAX_ARRAY_BYTES was introduced to allow setting the size for a free list of extra large buffers, thus allowing larger arrays to be transferred between clients and servers with matching configuration.


So that configuration option was meant to *allow* for larger arrays, never intended to *restrict* the array size.
If you do not set the variable, the largest buffer will still have a size of 16384 bytes.


Cheers,
Ralph


On Wed 26 May 2010 12:08:14 Matt Newville wrote:
Out of general curiosity, why is the default value for
EPICS_CA_MAX_ARRAY_BYTES set to 16384?

If I understand the purpose of this correctly, this parameter is meant
to prevent large data transfers over the CA protocol.  For the
discussion here, it does not limit the size of array in a real or soft
IOC -- the array data already exists, and people are unable to
transfer it unless they explicitly increase this value when they want
to transfer "large" array data.  Of course, once they set this high
enough, everything works, suggesting there is not a bandwidth issue.
That suggests the only effect of EPICS_CA_MAX_ARRAY_BYTES is to
prevent data that already exists from being transferred by CA.  Is
that an unfair characterization?

So, why is the default value not unlimited (or, say 2**32), so that
EPICS_CA_MAX_ARRAY_BYTES would have the role of allowing one to
explicitly set a maximum array size for data transfer,  instead of
being set  (as is currently the case) to make sure that existing,
desired data can be transferred from IOCs that have that data over
networks that can handle the load?

Thanks,

--Matt Newville <newville at cars.uchicago.edu> 630-252-0431


On Tue, May 25, 2010 at 6:36 PM, Till Straumann
<strauman@slac.stanford.edu> wrote:
On 05/25/2010 05:31 PM, Mark Rivers wrote:
No, there is no limit, just set EPICS_CA_MAX_ARRRAY_BYTES.  We are sending
16MB images over CA with no problem.

Amd 16K of doubles is not 1K values, it is 2K.  A double takes 8 bytes on
every architecture I know of.

IIRC it can be a little less because the EPICS_CA_MAX_ARRAY_BYTES include
the extra info (status, timestamp etc) -- details depend on the DBR type.

In labCA PVs are normally retrieved as DBR_TIME_xxx.

A limitation of the underlying ezca library is that when you fetch
units or limits (stuff that isn't provided with DBR_TIME_xxx) then
ezca uses a DBR_CTRL_xxx with the native element count of the PV
even though the array elements themselves are discarded.
This is unfortunate when you have a slow connection.

E.g., if you call

lcaGetUnits(pv)

and your PV has a large native element count then the entire
array is transferred and discarded (and the call will fail if
your EPICS_CA_MAX_ARRAY_BYTES is too small).

Luckily, if you ever run into that problem you can obviously
work around in most cases by reading the '.EGU' field

lcaGet(pv + '.EGU'); // this is scilab notation

HTH
T.
Mark


________________________________


From: tech-talk-bounces@aps.anl.gov on behalf of
emmanuel_mayssat@lynceantech.com
Sent: Tue 5/25/2010 4:15 PM
To: Till Straumann
Cc: EPICS Tech-Talk
Subject: Re: labCA and EPICS_CA_MAX_ARRAY_BYTES



Is there still a limit on the length of arrays?
I understand that the client and ioc needs to be configured with
EPICS_CA_MAX_ARRAY_BYTES

Note that the number of bytes is not the numbers of values.
16K of double variables is only 1K values....

is this still a limitation in 3.14.11?
--
E

On 10:27 Tue 25 May , Till Straumann wrote:

On 05/25/2010 09:55 AM, John Dobbins wrote:

Is it possible to use labCA to retrieve arrays larger than 16K?


Yes - I routinely use that.
You have to make sure EPICS_CA_MAX_ARRAY_BYTES is set
in the environment *before* you start matlab or scilab (RTM).

On the IOC, too, you need EPICS_CA_MAX_ARRAY_BYTES but
since you tested with caget that's obviously set up correctly.



Replies:
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville
References:
labCA and EPICS_CA_MAX_ARRAY_BYTES John Dobbins
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Till Straumann
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES emmanuel_mayssat
RE: labCA and EPICS_CA_MAX_ARRAY_BYTES Mark Rivers
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Till Straumann
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville

Navigate by Date:
Prev: Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville
Next: Re: CA web service Re: iPhone port Matthias Clausen
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017 
Navigate by Thread:
Prev: Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville
Next: Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017 
ANJ, 02 Sep 2010 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·