Subject: |
reading large data arrays over slow networks |
From: |
Matt Newville <[email protected]> |
To: |
EPICS Tech Talk <[email protected]> |
Date: |
Thu, 3 Nov 2011 15:08:54 -0500 |
Hi,
I'm seeing an issue with Channel Access getting "large data arrays"
and hope someone can provide some insight. First, this is not an
issue with EPICS_CA_MAX_ARRAY_BYTES, which is set large enough.
Rather the issue I'm seeing seems to raise the naive question:
Once I do a ca_array_get(type, count, chid, pvalue), when can I
successfully read the data in pvalue?
That is, how do I know how long to wait in ca_pend_event() and
ca_pend_io()? In most cases of a fast network and scalar values or
small arrays, values on the order of
ca_pend_event(1.e-3);
ca_pend_io(1.0);
seem to work well. But, if the array is "large" or the network
"slow", I have a hard time predicting these values, and accessing the
data may give incorrect values unless I've waited "long enough".
Using either a slow network or pend_event/io times that is clearly
"too short", say,
ca_pend_event(1.e-5);
ca_pend_io(0.01);
leaves the data at pvalue incorrect (all zeros). Trying to access
this data from python, I can get python to segfault for large enough
data (say, 4.2M ints). I'm not sure I fully understand why python is
crashing -- the segfault happens well after ca_array_get() returns,
but can happen in ca_pend_event() a short time later. If I save but
don't try to access the data in pvalue, or wait "long enough", it
seems I never have a crash.
Related questions are: What is supposed to happen if ca_pend_event()
and/or ca_pend_io() do time out? and: Is there a way to tell if
pend_event() or pend_io() have timed out or if there are events that
are pending?
I was hoping to be able to do something like
ret_ev = ca_pend_event(1.e-3);
ret_io = ca_pend_io(1.);
while (ret_io == ECA_TIMEOUT) {
ret_ev = ca_pend_event(1.e-3);
ret_io = ca_pend_io(1.);
}
Unfortunately, it seems that checking the return values are not so
helpful, as pend_event() always(?) returns ECA_TIMEOUT and pend_io()
seems to return ECA_TIMEOUT no more than one time before returning
ECA_NORMAL.
For what it's worth, I'm using base 3.14-12.1. Thanks in advance for
any insight.
--Matt Newville <newville at cars.uchicago.edu> 630-252-0431
- Replies:
- Re: reading large data arrays over slow networks Tim Mooney
- RE: reading large data arrays over slow networks Jeff Hill
- Re: reading large data arrays over slow networks Andrew Johnson
- Re: reading large data arrays over slow networks Kevin Peterson
- Navigate by Date:
- Prev:
Asyn or StreamDevice Error Messages: vxi11 clientIoCall errno Permission denied clnt_stat 5 William Lu
- Next:
Re: reading large data arrays over slow networks Tim Mooney
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
<2011>
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Asyn or StreamDevice Error Messages: vxi11 clientIoCall errno Permission denied clnt_stat 5 William Lu
- Next:
Re: reading large data arrays over slow networks Tim Mooney
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
<2011>
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|