Experimental Physics and
Industrial Control System

Matt Newville <[email protected]> · Mon, 19 Sep 2011 10:58:38 -0500

Hi Xiaoqiang, Emmanuel

Yes, thanks Xiaoqiang.  Actually, I played with this some over the
weekend, and found that exact point.  Here are a few more details --
I'll try a more complete writeup and post code for this over the next
couple days.

Connecting to 20K PVs (all connected, on several VME IOCs on the same
subnet), using a "normal" epics.PV approach:
    import epics
    <read pvnames>
    pvs = []
    for name in pvnames:
        pvs.append(epics.PV(name))

    vals = []
    for pv in pvs:
        vals.append(pv.get())

    <write values to disk>

takes about 11 seconds for me.  PVs use connection and event callbacks
internally.

Asserting that python was "the slow part", I went "straight to libca",
which I believe to be effectively equivalent to using C (the
libca.ca_*** calls go straight to the dll).  Here, a more complicated
example:

    import ctypes
    from epics import ca, dbr # used only for constants
    libca = ca.initialize_libca()
    <read pvnames>
    chidlist = []
    for name in pvnames:
        chid = ctypes.c_long()
        #  note: no connection callback
        ret = libca.ca_create_channel(name, 0, 0, 0, ctypes.byref(chid))
        chidlist.append(chid)

    ca.pend_event(1.e-3)
    ca.pend_io(1.0)

    for chid in chidlist:
        name = libca.ca_name(chid)
        count = libca.ca_element_count(chid)
        ftype = libca.ca_field_type(chid)
        pdat = (count*dbr.Map[ftype])()  # ok, a python lookup for type
        ret = libca.ca_array_get(ftype, count, chid, pdat)
        ca.pend_event(1.e-3)
        ca.pend_io(0.01)
        val = < code to convert pdat to python data type>

This *also* takes around 11 seconds, even though it uses a very
different model -- no connection callbacks, no subscription to
changes.  I checked the "convert pointer to python data" ... not the
culprit.

But, like Xiaoqiang pointed out, the pend_event() and pend_io() after
ca_array_get() are the culprit.  Replacing the "get loop" with two
loops (and adding more python code!) with:

    data = {}
    for chid in chidlist:
        name = libca.ca_name(chid)
        count = libca.ca_element_count(chid)
        ftype = libca.ca_field_type(chid)
        pdat = (count*dbr.Map[ftype])()  # ok, a python lookup for type
        ret = libca.ca_array_get(ftype, count, chid, pdat)
        data[name] = ftype, count, pdat
   ca.pend_event(1.e-3)
   ca.pend_io(1.0)
    for names in pvnames:
        ftype, count, pdat = data[name]
        val = < code to convert pdat to python data type>

cuts the run time to 2 seconds.   That is, replacing 20K pend_event();
pend_io() with 1 makes a huge difference.  I guess that's probably not
surprising to many people here.

FWIW, putting a connection callback back in to libca.create_channel()
seems to slow the code down to about 4 seconds.

OK, so here are my conclusions so far:
   a) Python is not  the slow part, at least not yet.  For the
original question of whether it is suitable for a backup tool, I'd say
it is.   I don't see any reason why it wouldn't be within a factor of
2 of the equivalent C code for 100K PVs.

   b) You need to be careful about every "pend()" call when trying to
get the best performance when connecting to many PVs.

I'm OK with changing pyepics ca layer so that connection callbacks are
optional and so that get() can  optionally return the
"pointer-to-dat", with polling for the data to show up.   That would
make it much easier to write code to "quickly connect to thousands of
PVs".

I'm not sure how to get this performance boost using higher-level PV
objects.  These use both connection callback and event callbacks,
which  I consider an important feature for ease-of-use.    It's a
reasonable goal to want to create thousands of PVs as quickly as
possible. I think that might imply an extra step was needed to setup
the callbacks. I'll have to think about how to do that  -- suggestions
welcome!

--Matt Newville <newville at cars.uchicago.edu> 630-252-0431

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System