EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  <20112012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  <20112012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: reading large data arrays over slow networks
From: Matt Newville <[email protected]>
To: EPICS Tech Talk <[email protected]>
Date: Thu, 3 Nov 2011 15:08:54 -0500
Hi,

I'm seeing an issue with Channel Access getting "large data arrays"
and hope someone can provide some insight.  First, this is not an
issue with EPICS_CA_MAX_ARRAY_BYTES, which is set large enough.
Rather the issue I'm seeing seems to raise the naive question:

    Once I do a ca_array_get(type, count, chid, pvalue), when can I
successfully read the data in pvalue?

That is, how do I know how long to wait in ca_pend_event() and
ca_pend_io()?    In most cases of a fast network and scalar values or
small arrays, values on the order of
   ca_pend_event(1.e-3);
   ca_pend_io(1.0);

seem to work well.   But, if the array is "large" or the network
"slow", I have a hard time predicting these values, and accessing the
data may give incorrect values unless I've waited "long enough".
Using either a slow network or pend_event/io times that is clearly
"too short",  say,
   ca_pend_event(1.e-5);
   ca_pend_io(0.01);

leaves the data at pvalue incorrect (all zeros).  Trying to access
this data from python, I can get python to segfault for large enough
data (say, 4.2M ints).  I'm not sure I fully understand why python is
crashing -- the segfault happens well after ca_array_get() returns,
but can happen in ca_pend_event() a short time later.  If I save but
don't try to access the data in pvalue, or wait "long enough", it
seems I never have a crash.

Related questions are:  What is supposed to happen if ca_pend_event()
and/or ca_pend_io() do time out?  and: Is there a way to tell if
pend_event() or pend_io() have timed out or if there are events that
are pending?

I was hoping to be able to do something like
   ret_ev = ca_pend_event(1.e-3);
   ret_io = ca_pend_io(1.);

   while (ret_io == ECA_TIMEOUT) {
      ret_ev = ca_pend_event(1.e-3);
      ret_io = ca_pend_io(1.);
   }

Unfortunately, it seems that checking the return values are not so
helpful, as pend_event() always(?) returns ECA_TIMEOUT and pend_io()
seems to return ECA_TIMEOUT no more than one time before returning
ECA_NORMAL.

For what it's worth, I'm using base 3.14-12.1.   Thanks in advance for
any insight.

--Matt Newville <newville at cars.uchicago.edu> 630-252-0431

Replies:
Re: reading large data arrays over slow networks Tim Mooney
RE: reading large data arrays over slow networks Jeff Hill
Re: reading large data arrays over slow networks Andrew Johnson
Re: reading large data arrays over slow networks Kevin Peterson

Navigate by Date:
Prev: Asyn or StreamDevice Error Messages: vxi11 clientIoCall errno Permission denied clnt_stat 5 William Lu
Next: Re: reading large data arrays over slow networks Tim Mooney
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  <20112012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: Asyn or StreamDevice Error Messages: vxi11 clientIoCall errno Permission denied clnt_stat 5 William Lu
Next: Re: reading large data arrays over slow networks Tim Mooney
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  <20112012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 18 Nov 2013 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·