Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017 
<== Date ==> <== Thread ==>

Subject: RE: JCA problems and questions
From: "Hu, Yong" <yhu@bnl.gov>
To: "'Matej Sekoranja'" <matej.sekoranja@cosylab.com>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Wed, 24 Oct 2012 16:57:51 +0000

Hello All,

 

I’d like to ask some questions related to TCP connection and automatic resizing of  max_array_bytes as well as show some of my thoughts.

 

1.  “100 connection * 10MB = 1GB”: This means the client is connected with 100 different Channel Access Servers (IOCs).

 

1) Suppose this client is CSS running on Linux.  How can we find out how many CA TCP connections are opened on the Linux machine? Is there a utility / plug-in in the CSS to provide some connection info just like the IOC shell command ‘casr’ which can report detailed info about the TCP connections on the CA Server side?

 

2) In this case, EPICS_CA_MAX_ARRAY_BYTES is set to 10MB on the CA client side. I did several tests to see the memory used by several clients “camonitor” on a Linux machine. The IOC (CA server) is one softIoc with some array-type records such as compress, waveform, subArray and aSub which have data size a little bit bigger than 16384 Bytes so that EPICS_CA_MAX_ARRAY_BYTES is set to 1 MB on the IOC side. I opened three terminals. One terminal is used to camonitor a scalar PV “camonitor yhuHost:aiExample”, one is used for “camonitor array-type-record” and another terminal is for the “top” and “free -m” commands to monitor the memory usage. The result from the “top” command seems interesting: the value of VIRT/RES for either “camonitor yhuHost:aiExample” or “camonitor array-type-record” is something like 11m / 2680 which means the virtual memory is around 10MB, but the actually used resident memory is much smaller. The “free -m” command shows the same thing: the used memory by those two “camonitor” is actually very small, not 2 connection * 10MB=20MB. So, my understanding of “100 connection * 10MB = 1GB” is that 1 GB is the total max. virtual memory size, but the actual physical memory consumed by those TCP connections,  which depends on the data size of array-type records on the IOCs’ sides, is usually less and smaller than 1GB.

 

Please note that my test setup is several CA clients connected to only one CA server, which seems reverse to the setup “100 connection * 10MB = 1GB” where only one CA client is connected to several CA servers. But I think these two setups should produce almost the same effects / results for testing memory usage by CA TCP connections.

 

2. “automatically resized up to max_array_bytes”: I agree with Matej that this is very convenient on the client side. I guess most of us have experienced having to setup EPICS_CA_MAX_ARRAY_BYTES to a bigger number on both IOC side (in st.cmd) and OPI side (i.e. CSS, EDM) when we deal with array-like records such as compress, waveform, subArray, aSub, etc. It would be nice to automatically (or semi-automatically) setup EPICS_CA_MAX_ARRAY_BYTES to an appropriate number on the IOC side during iocInit. I guess it might be feasible and here is what I think:

 

1) add something like iterateRecords(calculateMaxArrayBytes, NULL) at initDatabase() during iocBuild() to calculate the max. memory size (number of bytes) that an array-type record needs. For instance, if all records in the IOC are scalar-type,  we will do nothing with EPICS_CA_MAX_ARRAY_BYTES for this case. If there is a waveform record in the IOC, we can calculate the max. memory size (say max_array_mem_size) allocated for the waveform record by prec->nelm * dbValueSize(prec->FTVL). Use similar algorithms for any other array-type records to get their max. memory sizes and eventually we can get the max. value of max_array_mem_size. If (max_array_mem_size * 1.2)> 16384,  then we setup EPICS_CA_MAX_ARRAY_BYTES = max_array_mem_size * 1.2. The coefficient “1.2” (or other value) is for adding up additional bytes for compound data types (i.e. DBR_GR_DOUBLE).

 

2) the algorithm described above won’t work if the IOC contains non-standard (not included in the base) records such as records from the synApps package. In this case, we have to manually configure EPICS_CA_MAX_ARRAY_BYTES in the IOC st.cmd file. EPICS_CA_MAX_ARRAY_BYTES is defined before iocInit() so that iterateRecords(calculateMaxArrayBytes, NULL) just returns, does nothing.

 

3) So, iterateRecords(calculateMaxArrayBytes, NULL) will make EPICS_CA_MAX_ARRAY_BYTES transparent on the IOC side for most users in most cases. But manual configuration of  EPICS_CA_MAX_ARRAY_BYTES will eventually work for all cases.

 

Sorry for this long message. If most of us think it is worth implementing this kind of semi-automatic configuration of  EPICS_CA_MAX_ARRAY_BYTES on the IOC side, I will give a try when I have extra time.

 

Thanks,

 

Yong

 

 

From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Matej Sekoranja
Sent: Tuesday, October 23, 2012 4:33 PM
To: tech-talk@aps.anl.gov
Subject: Re: JCA problems and questions

 

Hi,

 

Rok Sabjan notified me about this thread. Thanks to Lewis for replies.

 

The old send buffer algorithm was to initialize the send buffer size to max_array_bytes and

automatically resize on demand (there is one send buffer per TCP connection). Not something

one would dare to use on a server, however very convenient on the client side.

 

However, if a client has a lot of connections there is a lot of memory required when max_array_bytes is large (e.g. 100 connection * 10MB = 1GB!).

Current algorithm starts with an initial size of 1k that can be automatically resized up to max_array_bytes.

This also mimics C++ CA algorithm (that has also evolved over the years).

 

Cheers,

Matej


Replies:
Re: JCA problems and questions Matej Sekoranja
References:
Re: JCA problems and questions Matej Sekoranja

Navigate by Date:
Prev: RE: Rotation image Widget Chen, Xihui
Next: Re: Trying to get NSLSII's version of CSS Michael Davidsaver
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017 
Navigate by Thread:
Prev: RE: JCA problems and questions Zhou, Jingchen
Next: Re: JCA problems and questions Matej Sekoranja
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017 
ANJ, 18 Nov 2013 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·