Experimental Physics and
Industrial Control System

"Hu, Yong" <[email protected]> · Tue, 10 Jun 2014 17:55:34 +0000

Hi Ralph,

Thanks for your detailed and helpful explanation. Now, I understand why Eric and Andrew thought "Having different clients behave differently seems like it’s going to cause a lot more problems than it solves.". I agree with all of you. 

Improvements on handling EPICS_CA_MAX_ARRAY_BYTES is really desired. I guess incorrect settings of EPICS_CA_MAX_ARRAY_BYTES causes most CA- / waveform-related problems. While we are talking about auto-configuration of EPICS_CA_MAX_ARRAY_BYTES, someone got trouble on his areaDetector driver because of incorrect settings of EPICS_CA_MAX_ARRAY_BYTES. See http://www.aps.anl.gov/epics/tech-talk/2014/msg00859.php  

Hope Andrew's approach "compiled-in default settings" will make things better.

Thanks to all for sharing your ideas,

Yong

________________________________________
From: Ralph Lange [[email protected]]
Sent: Friday, June 06, 2014 10:21 AM
To: EPICS Core-Talk
Cc: Hu, Yong
Subject: Re: auto-configure EPICS_CA_MAX_ARRAY_BYTES in catools/caget

Hello Yong,

Thanks for your contribution, but for the CA command line tools, I must
say I do not like the suggested patch.

The command line tools are intended to be the "swiss army knife" for
Channel Access, to be used in debugging problems in your CA setup, and
as standard reference when complex clients don't work as expected. The
convient end user tool before calling an expert and using wireshark, if
you will.

As a tool, they should offer as much information as possible, to help in
debugging and finding problems. As a reference, they should always try
to act exactly as other clients would, without adding features that
other clients do not implement.

E.g., if your blown-up fancy big graphics client is not showing that
large image (because you did not set EPICS_CA_MAX_ARRAY_BYTES large
enough), the command line tool should fail for exactly the same reason.
Otherwise (using the suggested patch), the command line tool would just
work, your fancy client would not, and the command line tool would push
you into believing your fancy client is buggy.

On the other hand, I would happily accept a patch that makes the command
line tool print out an error message, telling you that the data you are
trying to read is larger than EPICS_CA_MAX_ARRAY_BYTES. That would be
really useful, and exactly match the idea of a debugging aid.

Cheers,
~Ralph

On 06/06/2014 13:24, Hu, Yong wrote:
> Hi Andrew and Eric,
>
> Thanks for sharing your opinions and ideas. I have different views about the change I have made on caget.c and other things:
>
> 1. "Having different clients behave differently seems like it’s going to cause a lot more problems than it solves.". I don't think the change on the command-line tool caget will cause any problem. Firstly, different CA clients are actually behaving differently depending on how we define "behavior", i.e. users may feel that EDM runs so fast and consumes little CPU and memory resources while having opposite feelings when using CSS; users don't need to think about the settings of EPICS_CA_MAX_ARRAY_BYTES if their site-specific CSS already has this configured to a very large number, but they need to configure EPICS_CA_MAX_ARRAY_BYTES when they use the command caget on Windows; etc. Secondly, my proposed change doesn't change the CA APIs. It doesn't change any existing EPICS_CA_MAX_ARRAY_BYTES settings such as the one in ~/.profile or /etc/profile.d. Keep in mind that caget(.c) is just a very convenient command-line tool. Once it is executed, it is done and it is gone. Thirdly !
 but not lastly, if users already set EPICS_CA_MAX_ARRAY_BYTES to a very large number, you will find that my change on caget.c does nothing on EPICS_CA_MAX_ARRAY_BYTES if you look through the code closely. The code only reconfigures EPICS_CA_MAX_ARRAY_BYTES if and only if the value of EPICS_CA_MAX_ARRAY_BYTES is not big enough to get the job done successfully. In short, the change on caget does not introduce any bad behavior, it is a complement to the great command-line tool caget, it makes caget more convenient and ease-of-use.
>
> 2. "/etc/profile.d": you forget about the CA clients on Windows and Mac which doesn't have this thing. And sometimes it takes lots of effort to figure out how to configure EPICS_CA_MAX_ARRAY_BYTES on different Windows OSs.
>
> 3. I think it is very difficult to configure architecture-specific default settings. I saw some IOCs running linux-x86, but they have very limited memory.
>
> 4. We are in a big data era. But we still keep the default EPICS_CA_MAX_ARRAY_BYTES to a low value at 16K bytes mainly because there are some tiny IOCs which have limited memory resource. I believe that most modern CA clients such as EDM, CSS, etc. should have have no problem with EPICS_CA_MAX_ARRAY_BYTES to tens of or even hundreds of Megabytes. How about the idea of creating another environment variable "EPICS_CAS_MAX_ARRAY_BYTES" with a default low value at 16K dedicated for CA server (IOC) and using EPICS_CA_MAX_ARRAY_BYTES with a default high value for CA client? The configurations of max_array_bytes on the client side and on the server side are independent, but currently we are using a single environment variable for two independent settings. Having said that, introducing a new environment variable may introduce other problems.
>
> 5. Any progress on adding new parameters dynamically: https://blueprints.launchpad.net/epics-base/+spec/epicsconf? Will it solve the problems related to EPICS_CA_MAX_ARRAY_BYTES?
>
> 6. How about the idea of auto-configuring of EPICS_CA_MAX_ARRAY_BYTES on the server side?
>
> Jeff, Ralph: you are the creators of CA APIs & catools. What do you think?
>
> Thanks,
>
> Yong
>
> ________________________________________
> From: [email protected] [[email protected]] on behalf of Andrew Johnson [[email protected]]
> Sent: Thursday, June 05, 2014 5:09 PM
> To: [email protected]
> Subject: Re: auto-configure EPICS_CA_MAX_ARRAY_BYTES in catools/caget
>
> On 06/05/2014 03:56 PM, Eric Norum wrote:
>> I’m not sure that this is a good idea.  Having different clients
>> behave differently seems like it’s going to cause a lot more problems
>> than it solves.
> I agree; the best solution for most modern clients should be to set
> EPICS_CA_MAX_ARRAY_BYTES to a very large number in a script that gets
> installed into /etc/profile.d so everybody gets that setting without
> having to think about it. About the only place where you would want to
> avoid doing that would be in embedded IOCs that don't have virtual memory.
>
> You could also put that big number in the configure/CONFIG_SITE_ENV file
> of the copy of Base you use for client applications, as long as you use
> a different version of Base without that setting for building your
> embedded IOCs.
>
> If someone wants to work out how to configure architecture-specific
> default settings during the build of Base that might be an interesting
> patch for future releases.
>
> - Andrew
>
>
>> On Jun 5, 2014, at 11:40 AM, Hu, Yong <[email protected]> wrote:
>>
>>> Hello All,
>>>
>>> I guess many EPICS users, especially beginners, had problems when
>>> using the command "caget" to read large waveform data because they
>>> forgot to reconfigure EPICS_CA_MAX_ARRAY_BYTES to a bigger number
>>> or they didn't know how to configure this parameter. Now, this
>>> problem seems to be fixed. See attached patch or .c file. I tested
>>> it on Debian 6 / gcc 4.4.5 / base-3.14.12.2.
>>>
>>> I hope no users would use the command "camonitor" for big waveform
>>> data. Auto-configuring of EPICS_CA_MAX_ARRAY_BYTES in camonitor
>>> seems more complicated.
>>>
>>> Two years ago, there were discussions about automatically resizing
>>> max_array_bytes:
>>> http://www.aps.anl.gov/epics/tech-talk/2012/msg02136.php . I have
>>> an idea about how to implement this on the ioc side (RSRV), but no
>>> progress has been made yet. We just successfully completed NSLS-II
>>> Storage Ring phase-I commissioning and our machine is in
>>> maintenance. So, I have some free time to work on this again.
>>>
>>> All are welcomed to test the attached caget on Windows, Mac, and
>>> other Linux distros. Any idea & suggestion is welcomed!
>>>
>>> Yong<caget.c-3.14.12.2.patch><caget.c>

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System