Hi Matt,
On Thursday 27 May 2010 20:25:01 Matt Newville wrote:
>
> Yes, I understand this. Obviously, we're talking about transferring
> data arrays which already exist on real IOCs, on networks that can
> transfer them. The advice of "increase EPICS_CA_MAX_ARRAY_BYTES and
> everything will work fine" implies (perhaps even demonstrates) that
> the default value of EPICS_CA_MAX_ARRAY_BYTES is too small.
The CA server gets one chance to make a complete copy of each array as part of
the process of sending it to its clients, so it must allocate a buffer big
enough to hold that array. When a new client connects the server allocates
two buffers of 16K each to that client (for send and receive), but when a
request comes in for something bigger it attempts to upgrade the relevant
buffer to MAX_ARRAY_BYTES plus a bit more to cover the header overhead. Some
of our vxWorks IOCs at the APS can (and do) have up to 250 clients, so they
need 250*2*16K ~= 8MB of RAM just for default-sized CA buffers. That's not a
problem on a workstation class machine where the client count could be even
larger if necessary, but some of our vxWorks IOCs have only about 32MB of RAM
total.
> > This is especially important for servers, since IOCs may have limited
> > memory and high availability requirements.
>
> Are there IOCs that cannot have 16k of PV data values? The value
> cannot be set lower than 16k, so the limited-memory argument is a bit
> suspect. Anyway, the issue that keeps cropping up is not whether IOCs
> can generate and serve the data, but whether clients can receive them.
The ColdFire IOCs that APS is using for BPMs and other FPGA-implemented
processing have 16MB of RAM. They obviously can't support as many clients as
the 250 I mentioned above, and in many cases they also have one or more
waveform records holding large acquisition history data buffers, so they have
to have MAX_ARRAY_BYTES set to something larger than the default.
The standard APS Unix and Linux workstation login scripts have the environment
variable EPICS_CA_MAX_ARRAY_BYTES set to 140000 which we found to be suitable
for our systems. Could you adopt a similar approach for your client machines?
> Is there really no technical reason for this value? If it is only
> historical, and it can be changed with a one-line fix to
> base/configure/CONFIG_ENV, why not change it?
16K seems to be about right for a minimum buffer size from experience, I don't
recall anyone complaining that it was too big, although before Jeff added the
MAX_ARRAY_BYTES support it was also the maximum buffer size and that caused
many complaints. If you feel the default should be smaller I'm sure Jeff Hill
will be happy to discuss that point with you (but not until he gets back to
LANL, as he can't check his email while he's over here in Europe).
> It appears to me that
> the value is really used only in base/src/ca/cac.cpp (starting around
> line 185), and there only to set a maximum data size. There, you'll
> see two interesting pieces of information:
> a) setting EPICS_CA_MAX_ARRAY_BYTES < 16384 (MAX_TCP) will result in
> it being reset to 16384:
> so that one cannot set this to a number smaller than 16384 (so
> much for allowing one to conserve
> resources on memory-starved systems!!)
> b) room for header information is added onto this value (apparently
> contradicting an earlier description),
> up to 2*32 bytes.
Yes, we were getting complaints from people who didn't read the documentation,
and there's only so many times you can write RTFM in email replies, so Jeff
added the header size so the name reflects reality.
> > So that configuration option was meant to *allow* for larger arrays,
> > never intended to *restrict* the array size.
>
> But restricting is exactly the effect that it does have for client
> programs! The array already exists in some IOC, which is prepared to
> transfer the data to any and all clients. But unless a client
> application knows how large an array is **BEFORE THE APPLICATION
> BEGINS USING CA**, it cannot get the data.
Or you just have some default environment variable (or default set in
CONFIG_SITE_ENV) for all workstation users which is big enough for everything
you think they'll need.
HTH,
- Andrew
--
The best FOSS code is written to be read by other humans -- Harald Welte
- References:
- labCA and EPICS_CA_MAX_ARRAY_BYTES John Dobbins
- Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Ralph Lange
- Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville
- Navigate by Date:
- Prev:
Re: EPICS wiki is down Andrew Johnson
- Next:
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Ralph Lange
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
<2010>
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Matt Newville
- Next:
Re: labCA and EPICS_CA_MAX_ARRAY_BYTES Ralph Lange
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
<2010>
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|