Experimental Physics and
Industrial Control System

"Abbott, Michael (DLSLtd,RAL,DIA)" <[email protected]> · Tue, 29 Sep 2009 09:45:59 +0100

A little while ago I was asked to comment on the variety of available
Python channel access bindings, and I composed the survey below;  I've
been asked to broadcast this more widely to the tech-talk list, so here
it is.  For a bit of context, I am the developer of the Diamond
cothread.catools interface, so I can't help promoting it below, but my
impressions of EpicsCA are also good -- my only real criticism is the
lack of asychronous support (though I think cothread does meta-data
better).

I hadn't realised just how tangled the history of Python channel access
interfaces has become!  There's a very nice survey on the PSI site at

http://controls.web.psi.ch/cgi-bin/twiki/view/Main/NewPythonEpicsInterfa
ce

I think we can rule the old FNAL CaPython out of consideration.  After 
closer investigation, it appears that NPEI and PythonCA are virtually 
identical, and the only differences that I can see are in the setup 
scripts, particularly as the PSI NPEI is designed as a patch to KEK's 
PythonCA which appears to have already been largely incorporated.

So this means that we need to understand the relationships between these
three libraries:

    PythonCA(KEK) / NPEI(PSI)
http://www-acc.kek.jp/EPICS_Gr/products.html
http://controls.web.psi.ch/cgi-bin/twiki/view/Main/NewPythonEpicsInterfa
ce

    EpicsCA (UChicago)
http://cars9.uchicago.edu/~newville/Epics/Python/

    Cothread (Diamond)
http://controls.diamond.ac.uk/downloads/python/cothread/

I have to confess this seems a crazy situation, and I guess I've made
things worse by adding the cothread library to the mix.

How are we to compare these three?  They all seem actively supported and
developed, so I can think of the following critera:

1. API.  How comfortable is the API to use, how complete is it, is it
well documented, and how well behaved is it?

2. Implementation technology, dependencies, quality of code, etc.  A
quick inspection of the code can give some clues about maintainability
and possible installation interactions.  

3. Quality of implementation -- how well does the library behave?  This
requires some experience using the library, so unfortunately I can't
make any comment on the libraries other than cothread in this area.

So first comments on the API.

Both EpicsCA and Cothread are inspired by EZCA, and indeed cothread
sticks much more closely to the original inspiration with the complete
cothread catools implementation provided through three functions: caget,
caput and camonitor.  Cothread has extended the capabilities of these
functions while keeping to the original motivation, which was to make
channel access as easy to use as possible.

The PythonCA appears to be a relatively thin wrapper around the channel
access library, and exposes a more complicated interface that is quite a
bit harder to use.  

I think it is fair to say that EpicsCA has a more straightforward and
naive implementation than cothread, and provides a somewhat less
functional interface.  For example, EpicsCA forces explicit polling of
the CA library in user code.

So in summary, PythonCA is a bit too low level to be comfortable to use,
while both EpicsCA and Cothread wrap the low level details.  I think
Cothread is both more complete and smoother to use -- but having
designed it, I am biased here!

Implementation Dependencies

Cothread depends on ctypes, numpy and greenlet.  The numpy library is
used for both waveform support (not as well supported by the other
libraries, I believe) and to assist with data type conversion.  The
dependency on greenlet is the most awkward, but this is a very tiny
component that is easy enough to compile on all important targets.

EpicsCA depends on SWIG, and can use numpy if it is available.

PythonCA has been implemented largely as a C extension, so needs to be
compiled but has no dependency on SWIG.  There is a reference to numpy
in the source code, but I think it's an unused relic.

Threading Issues

The main important implementation issue where the three libraries differ
substantially is in the handling of threading.  In summary:

EpicsCA:
    Does not support any form of threading, and the EPICS CA library
    needs to be explicitly polled by user code.

PythonCA:
    Uses standard Python threads, and runs with preemptive callbacks
    enabled.

Cothread:
    Uses coroutines instead of standard threads and polls the EPICS CA
    library; however in this case the polling is concealed from user
code.

The argument between standard threads and cothreads is rather delicate,
and was covered in some detail in the posting referenced below:
    http://www.aps.anl.gov/epics/tech-talk/2009/msg00227.php 
(See the section 'why "cothread"?')

In an early version of the cothread library we compared polled versus
preemptive callbacks and found that polling was quite a bit more
efficient when processing many camonitor updates, but this test should
probably be repeated.  Certainly cothread works fine with monitoring
several hundred PVs, each updating many times a second.

Cothread

Finally some comments on the cothread library.  Note that this doesn't
use SWIG, instead the ctypes library (now built into Python) allows C
functions to be called directly from Python without any compilation
required.  It's designed to be simple to use, but I must flag a point
that may need attention.

Note that cothreads are not classic (asynchronous) threads, instead
they're sometimes referred to as microthreads.  For most users the
distinction will be too subtle to be relevant, but there are a couple of
places where this will be visible.  Firstly, calling a blocking cothread
library function method (eg a Wait method) from another (classic) thread
will, in the current version of the cothread library, cause unexpected
problems.  I expect I can fix this, but the unwary programmer may have a
puzzling surprise.  The ThreadedEventQueue class is provided to help
with this issue.

Secondly, and but also fixable, there is an awkwardness in the
underlying implementation of coroutines provided by the greenlet
library: it currently works by swapping the entire active stack frame in
place (rather than by creating a separate stack for each cothread /
greenlet / coroutine / microthread, which is the more usual approach).
This doesn't disturb the operation of Python, but C extensions which
call into Python can be disrupted by this (this limits Qt4 integration,
for example).  I'd quite like to fix this by providing our own coroutine
library to replace greenlet, making it allocate a fresh stack for each
cothread.

I hope this is helpful.

    Michael Abbott

--
I apologise, I have no control over the stuff below.
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System