Experimental Physics and
Industrial Control System

Could be a limitation of the CA protocol

ca/caProto.h:

typedef struct ca_hdr {
    ca_uint16_t m_cmmd;         /* operation to be performed */
    ca_uint16_t m_postsize;     /* size of payload */
    ca_uint16_t m_dataType;     /* operation data type */
    ca_uint16_t m_count;        /* operation data count */
    ca_uint32_t m_cid;          /* channel identifier */
    ca_uint32_t m_available;    /* protocol stub dependent */
} caHdr;

but I don't know if there are any hacks in the protocol
to communicate arrays larger than 64k. In this case
it would be something else hitting the ceiling.

HTH
-- Till

On 01/11/2011 01:47 PM, Mark Rivers wrote:

Folks,

There appears to be a serious problem with EPICS Channel Access when using large arrays in 3.14.12.

This is easy to demonstrate.  Simply load a waveform record with more than 64K elements, and then use cainfo to report on the record.  It does not matter what version of cainfo is used, 3.14.12, 3.14.11, 3.14.10 and 3.14.8.2 all show the same thing.  The problem is in the server.

I just built the example application from base, and loaded a 1 record database.

Here is the database:

corvette:example/iocBoot/iocexample>more test.template
record(waveform, "TEST:waveform") {
    field(FTVL, "UCHAR")
    field(NELM, "2000000")
}

So it is a waveform record, type UCHAR, 2 million elements.

I have set EPICS_CA_MAX_ARRAY_BYTES to 20000000 (20 MB) on both client and server.

When I run the IOC and do caget on the NELM field of the waveform record I get the correct value: corvette:areaDetector/iocBoot/iocProsilica>caget TEST:waveform.NELM

TEST:waveform.NELM             2e+06

However, when I run cainfo I get 33920 for the element count:

corvette:areaDetector/iocBoot/iocProsilica>cainfo TEST:waveform

TEST:waveform
     State:         connected
     Host:          corvette.cars.aps.anl.gov:48884
     Access:        read, write
     Data type:     DBR_CHAR (native: DBF_CHAR)
     Element count: 33920

33920 is 2000000 modulo 65536.

I then did a systematic study and found that cainfo reports the correct element count up to 65535, but rolls over to 0 at 65536.

Load the database with NELM=65535
corvette:areaDetector/iocBoot/iocProsilica>caget TEST:waveform.NELM
TEST:waveform.NELM             65535
corvette:areaDetector/iocBoot/iocProsilica>cainfo TEST:waveform
TEST:waveform
     State:         connected
     Host:          corvette.cars.aps.anl.gov:5064
     Access:        read, write
     Data type:     DBR_CHAR (native: DBF_CHAR)
     Element count: 65535

Load the database with NELM=65536
corvette:areaDetector/iocBoot/iocProsilica>caget TEST:waveform.NELM
TEST:waveform.NELM             65536
corvette:areaDetector/iocBoot/iocProsilica>cainfo TEST:waveform
TEST:waveform
     State:         connected
     Host:          corvette.cars.aps.anl.gov:5064
     Access:        read, write
     Data type:     DBR_CHAR (native: DBF_CHAR)
     Element count: 0

It thus appears that the high word for the native element count is set to 0 in 3.14.12.

This means that applications that send large arrays over Channel Access (e.g. areaDetector viewers) will not work in 3.14.12.

Interestingly the CAJ native Java library does not have a problem, and the ImageJ viewer in areaDetector does work.

Andrew Johnson is aware of the problem, so hopefully there will be a fix soon. :-)

Mark

________________________________________

From: Mark Rivers
Sent: Mon 1/10/2011 9:31 PM
To: [email protected]
Cc: [email protected]
Subject: FW: EPICS problem?
Hi Jeff,

I am having a problem with EPICS 3.14.12 that could be a Channel Access problem. I am forwarding a message I sent to Andrew earlier this evening. The problem happens on both Linux and Win32 IOCs. The problem is that CA clients are reporting that the element count in a large array (NELM=4177920) is only 49152 bytes, which is incorrect. The problem is not related to the version of the client: it happens with "cainfo" from 3.14.12, 3.14.11, 3.14.10 and 3.14.8.2. It appears to be a problem with the server in 3.14.12? This is what I see with the server built with 3.14.12: corvette:ADApp/op/adl>/usr/local/epics/base-3.14.12/bin/linux-x86/cainfo 13PS1:image1:ArrayData

13PS1:image1:ArrayData
     State:            connected
     Host:             corvette.cars.aps.anl.gov:5064
     Access:           read, write
     Native data type: DBF_CHAR
     Request type:     DBR_CHAR
     Element count:    49152
When I run essentially the same IOC built with 3.14.10 I do not have this problem, cainfo (from 3.14.12) shows me the correct number of elements.  This is what I see with the server built with 3.14.10:

corvette:ADApp/op/adl>/usr/local/epics/base-3.14.12/bin/linux-x86/cainfo 13PS1:image1:ArrayData

13PS1:image1:ArrayData
     State:            connected
     Host:             corvette.cars.aps.anl.gov:5064
     Access:           read, write
     Native data type: DBF_CHAR
     Request type:     DBR_CHAR
     Element count:    4177920

Interestingly Java clients using the native Java CA library can read the entire array, while clients built with the C library seem only to be able to get the first 49152 bytes. In both cases NELM is 4177920: corvette:ADApp/op/adl>/usr/local/epics/base-3.14.12/bin/linux-x86/caget 13PS1:image1:ArrayData.NELM

13PS1:image1:ArrayData.NELM    4.17792e+06

The server and the client both have EPICS_CA_MAX_ARRAY_BYTES = 20MB

epics>  epicsPrtEnvParams
...
EPICS_CA_MAX_ARRAY_BYTES: 20000000
corvette:ADApp/op/adl>echo $EPICS_CA_MAX_ARRAY_BYTES
20000000
Any ideas?
Thanks,
Mark

________________________________________
From: Mark Rivers
Sent: Mon 1/10/2011 7:01 PM
To: 'Andrew Johnson'
Cc: 'Beamline Controls and Data Acquisition'
Subject: EPICS problem?
Hi Andrew,

I am having a strange problem with areaDetector in 3.14.12.  The image is a large waveform record:

epics>  dbpr 13PS1:image1:ArrayData 5

ACKS: NO_ALARM ACKT: YES APST: Always ASG: ASP: (nil) BKPT: 00 BPTR: 0xb1c03008 BUSY: 0 DESC: DISA: 0 DISP: 0 DISS: NO_ALARM DISV: 1 DPVT: 0xb192c318 DSET: 0x884b314 DTYP: asynInt8ArrayIn EGU: EVNT: 0 FLNK:CONSTANT 0 FTVL: UCHAR HASH: 0 HOPR: 0 INP:INST_IO @asyn(Image1,0,1)STD_ARRAY_DATA LCNT: 0 LOPR: 0 LSET: 0xb193ede0 MLIS: 00 00 00 00 00 00 00 00 00 00 00 00 MLOK: d8 e4 01 b2 MPST: Always NAME: 13PS1:image1:ArrayData NELM: 4177920 NORD: 1392640 NSEV: NO_ALARM NSTA: NO_ALARM PACT: 0 PHAS: 0 PINI: NO PPN: (nil) PPNR: (nil) PREC: 0 PRIO: LOW PROC: 0 PUTF: 0 RARM: 0 RDES: 0x97d00d0 RPRO: 0 RSET: 0x884c660 SCAN: I/O Intr SDIS:CONSTANT SEVR: NO_ALARM SIML:CONSTANT SIMM: NO SIMS: NO_ALARM SIOL:CONSTANT SPVT: 0xb1602ab8 STAT: NO_ALARM TIME: 2011-01-10 18:54:13.484892926 TPRO: 0 TSE: 0 TSEL:CONSTANT UDF: 0 VAL: (nil)

So NELM is 4177920.

This is epicsPrtEnvParams on the IOC:
epics>  epicsPrtEnvParams
EPICS_AR_PORT: 7002
EPICS_CAS_AUTO_BEACON_ADDR_LIST is undefined
EPICS_CAS_BEACON_ADDR_LIST is undefined
EPICS_CAS_BEACON_PERIOD is undefined
EPICS_CAS_BEACON_PORT is undefined
EPICS_CAS_IGNORE_ADDR_LIST is undefined
EPICS_CAS_INTF_ADDR_LIST is undefined
EPICS_CAS_SERVER_PORT is undefined
EPICS_CA_ADDR_LIST is undefined
EPICS_CA_AUTO_ADDR_LIST: YES
EPICS_CA_BEACON_PERIOD: 15.0
EPICS_CA_CONN_TMO: 30.0
EPICS_CA_MAX_ARRAY_BYTES: 20000000
EPICS_CA_MAX_SEARCH_PERIOD: 300.0
EPICS_CA_NAME_SERVERS is undefined
EPICS_CA_REPEATER_PORT: 5065
EPICS_CA_SERVER_PORT: 5064
EPICS_CMD_PROTO_PORT is undefined
EPICS_IOC_LOG_FILE_COMMAND is undefined
EPICS_IOC_LOG_FILE_LIMIT: 10000000
EPICS_IOC_LOG_FILE_NAME: /home/epics/logs/IOC.log
EPICS_IOC_LOG_INET is undefined
EPICS_IOC_LOG_PORT: 7004
EPICS_TIMEZONE: CUS::360:031302:110602
EPICS_TS_NTP_INET is undefined
IOCSH_HISTSIZE: 50

So EPICS_CA_MAX_ARRAY_BYTES is 20MB, which is plenty large enough.

However, this is what cainfo reports:
corvette:ADApp/op/adl>/usr/local/epics/base-3.14.12/bin/linux-x86/cainfo 13PS1:image1:ArrayData
13PS1:image1:ArrayData
     State:            connected
     Host:             corvette.cars.aps.anl.gov:47430
     Access:           read, write
     Native data type: DBF_CHAR
     Request type:     DBR_CHAR
     Element count:    49152

Note that it reports Element count - 49152, which is completely wrong.

Caget reports the same thing:
corvette:ADApp/op/adl>/usr/local/epics/base-3.14.12/bin/linux-x86/caget 13PS1:image1:ArrayData | more
13PS1:image1:ArrayData 49152 32 32 31 32 32 33 32 33 32 33 32 31 31 31 31 32 32 32 31 31 32 33 32 33 33 33 33 33 33 33 33 32 32 33 33 32 32 32 33 33 34 33 33 33 33 32 31 31
  32 32 31 31 32 31 31 30 31 29 30 30 30 28 28 26 27 26 26 23 24 2

If I try to read the array in IDL I only get 49152 bytes.  But if I read it in Java, using my ImageJ plugin, I get the entire array.

Something is wrong!

Mark

Replies:

RE: Problem using large arrays with Channel Access in EPICS 3.14.12 Mark Rivers

Re: Problem using large arrays with Channel Access in EPICS 3.14.12 Andrew Johnson

References:

Problem using large arrays with Channel Access in EPICS 3.14.12 Mark Rivers

Navigate by Date:

Prev: Problem using large arrays with Channel Access in EPICS 3.14.12 Mark Rivers

Next: RE: Problem using large arrays with Channel Access in EPICS 3.14.12 Mark Rivers

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  <2011>  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: Problem using large arrays with Channel Access in EPICS 3.14.12 Mark Rivers

Next: RE: Problem using large arrays with Channel Access in EPICS 3.14.12 Mark Rivers

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  <2011>  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024