1.  “100 connection * 10MB = 1GB”: This means the client is connected with 100 different Channel Access Servers (IOCs).

 

1) Suppose this client is CSS running on Linux.  How can we find out how many CA TCP connections are opened on the Linux machine? Is there a utility / plug-in in the CSS to provide some connection info just like the IOC shell command ‘casr’ which can report detailed info about the TCP connections on the CA Server side?

 

2) In this case, EPICS_CA_MAX_ARRAY_BYTES is set to 10MB on the CA client side. I did several tests to see the memory used by several clients “camonitor” on a Linux machine. The IOC (CA server) is one softIoc with some array-type records such as compress, waveform, subArray and aSub which have data size a little bit bigger than 16384 Bytes so that EPICS_CA_MAX_ARRAY_BYTES is set to 1 MB on the IOC side. I opened three terminals. One terminal is used to camonitor a scalar PV “camonitor yhuHost:aiExample”, one is used for “camonitor array-type-record” and another terminal is for the “top” and “free -m” commands to monitor the memory usage. The result from the “top” command seems interesting: the value of VIRT/RES for either “camonitor yhuHost:aiExample” or “camonitor array-type-record” is something like 11m / 2680 which means the virtual memory is around 10MB, but the actually used resident memory is much smaller. The “free -m” command shows the same thing: the used memory by those two “camonitor” is actually very small, not 2 connection * 10MB=20MB. So, my understanding of “100 connection * 10MB = 1GB” is that 1 GB is the total max. virtual memory size, but the actual physical memory consumed by those TCP connections,  which depends on the data size of array-type records on the IOCs’ sides, is usually less and smaller than 1GB.

Please note that my test setup is several CA clients connected to only one CA server, which seems reverse to the setup “100 connection * 10MB = 1GB” where only one CA client is connected to several CA servers. But I think these two setups should produce almost the same effects / results for testing memory usage by CA TCP connections.

 

2. “automatically resized up to max_array_bytes”: I agree with Matej that this is very convenient on the client side. I guess most of us have experienced having to setup EPICS_CA_MAX_ARRAY_BYTES to a bigger number on both IOC side (in st.cmd) and OPI side (i.e. CSS, EDM) when we deal with array-like records such as compress, waveform, subArray, aSub, etc. It would be nice to automatically (or semi-automatically) setup EPICS_CA_MAX_ARRAY_BYTES to an appropriate number on the IOC side during iocInit. I guess it might be feasible and here is what I think:

 

1) add something like iterateRecords(calculateMaxArrayBytes, NULL) at initDatabase() during iocBuild() to calculate the max. memory size (number of bytes) that an array-type record needs. For instance, if all records in the IOC are scalar-type,  we will do nothing with EPICS_CA_MAX_ARRAY_BYTES for this case. If there is a waveform record in the IOC, we can calculate the max. memory size (say max_array_mem_size) allocated for the waveform record by prec->nelm * dbValueSize(prec->FTVL). Use similar algorithms for any other array-type records to get their max. memory sizes and eventually we can get the max. value of max_array_mem_size. If (max_array_mem_size * 1.2)> 16384,  then we setup EPICS_CA_MAX_ARRAY_BYTES = max_array_mem_size * 1.2. The coefficient “1.2” (or other value) is for adding up additional bytes for compound data types (i.e. DBR_GR_DOUBLE).

 

2) the algorithm described above won’t work if the IOC contains non-standard (not included in the base) records such as records from the synApps package. In this case, we have to manually configure EPICS_CA_MAX_ARRAY_BYTES in the IOC st.cmd file. EPICS_CA_MAX_ARRAY_BYTES is defined before iocInit() so that iterateRecords(calculateMaxArrayBytes, NULL) just returns, does nothing.

 

3) So, iterateRecords(calculateMaxArrayBytes, NULL) will make EPICS_CA_MAX_ARRAY_BYTES transparent on the IOC side for most users in most cases. But manual configuration of  EPICS_CA_MAX_ARRAY_BYTES will eventually work for all cases.

 

Sorry for this long message. If most of us think it is worth implementing this kind of semi-automatic configuration of  EPICS_CA_MAX_ARRAY_BYTES on the IOC side, I will give a try when I have extra time.

 

Thanks,

 

Yong

 

 

From: [email protected] [mailto:[email protected]] On Behalf Of Matej Sekoranja

Sent: Tuesday, October 23, 2012 4:33 PM
To: [email protected]
Subject: Re: JCA problems and questions

 

Hi,

 

Rok Sabjan notified me about this thread. Thanks to Lewis for replies.

 

The old send buffer algorithm was to initialize the send buffer size to max_array_bytes and

automatically resize on demand (there is one send buffer per TCP connection). Not something

one would dare to use on a server, however very convenient on the client side.

 

However, if a client has a lot of connections there is a lot of memory required when max_array_bytes is large (e.g. 100 connection * 10MB = 1GB!).

Current algorithm starts with an initial size of 1k that can be automatically resized up to max_array_bytes.

This also mimics C++ CA algorithm (that has also evolved over the years).

 

Cheers,

Matej