Experimental Physics and
Industrial Control System

John Hammonds <[email protected]> · Tue, 29 Sep 2009 09:41:01 -0500

Mike,

I have done quite a bit work to update the FNAL CaPython/CaChannel. I

did a presentation on this at the Collaboration meeting in Italy last

fall

(http://agenda.infn.it/contributionDisplay.py?contribId=33&amp;confId=715).

I have done this trying to keep the intent of the old CaChannel/CaPython

from Geoff Savage. I was preparing a web page on this back in the

winter but got sidetracked by some other work. One of the main problems

that I had was that this relies on SWIG to wrap the c libraries. During

the last update of SWIG many of the original concepts were just blown

out of the water and this caused a major overhaul of the wrap. Now that

this overhaul is done though, I believe that this is actually a viable

candidate for the future.

The new version has been tested and is being used at the APS. This

version is compatible with SWIG 1.3 and has been tested with Python 2.4

and 2.5, hopefully this winter we will look at 2.6. It is a more or

less complete wrap of the C channel access library (some functions I

have not used so they have not seen much if any battle testing).

Callbacks in python are supported. I have done my best to support all

of the DBR (DBR_XXXX, DBR_CTRL_XXXX, DBR_GR_XXXX, ...). To date this

really only implements the Channel Access 3.12 library (no threading)

but when compiled against 3.14 large array support is available. I

believe that this is one of the more complete Python Channel access

libraries available since this was done as a more or less 1->1 map of

the channel accees library. CaChannel is meant to smooth out some of

the rough edges for the average Joe Programmer.

I will do my best to get the web page up and running soon. The APS is

due to turn on next week but once I will make this a priority once

things settle a bit. I would also like to get the multi-threading stuff

done soon as I have some need for this.

John

Abbott, Michael (DLSLtd,RAL,DIA) wrote:

A little while ago I was asked to comment on the variety of available
Python channel access bindings, and I composed the survey below;  I've
been asked to broadcast this more widely to the tech-talk list, so here
it is.  For a bit of context, I am the developer of the Diamond
cothread.catools interface, so I can't help promoting it below, but my
impressions of EpicsCA are also good -- my only real criticism is the
lack of asychronous support (though I think cothread does meta-data
better).

I hadn't realised just how tangled the history of Python channel access
interfaces has become!  There's a very nice survey on the PSI site at

http://controls.web.psi.ch/cgi-bin/twiki/view/Main/NewPythonEpicsInterfa

ce

I think we can rule the old FNAL CaPython out of consideration. After

closer investigation, it appears that NPEI and PythonCA are virtually

identical, and the only differences that I can see are in the setup

scripts, particularly as the PSI NPEI is designed as a patch to KEK's

PythonCA which appears to have already been largely incorporated.

So this means that we need to understand the relationships between these
three libraries:

    PythonCA(KEK) / NPEI(PSI)
http://www-acc.kek.jp/EPICS_Gr/products.html
http://controls.web.psi.ch/cgi-bin/twiki/view/Main/NewPythonEpicsInterfa
ce

    EpicsCA (UChicago)
http://cars9.uchicago.edu/~newville/Epics/Python/

    Cothread (Diamond)
http://controls.diamond.ac.uk/downloads/python/cothread/

I have to confess this seems a crazy situation, and I guess I've made
things worse by adding the cothread library to the mix.

How are we to compare these three?  They all seem actively supported and
developed, so I can think of the following critera:

1. API.  How comfortable is the API to use, how complete is it, is it
well documented, and how well behaved is it?

2. Implementation technology, dependencies, quality of code, etc.  A
quick inspection of the code can give some clues about maintainability

and possible installation interactions.

3. Quality of implementation -- how well does the library behave?  This
requires some experience using the library, so unfortunately I can't
make any comment on the libraries other than cothread in this area.

So first comments on the API.

Both EpicsCA and Cothread are inspired by EZCA, and indeed cothread
sticks much more closely to the original inspiration with the complete
cothread catools implementation provided through three functions: caget,
caput and camonitor.  Cothread has extended the capabilities of these
functions while keeping to the original motivation, which was to make
channel access as easy to use as possible.

The PythonCA appears to be a relatively thin wrapper around the channel
access library, and exposes a more complicated interface that is quite a

bit harder to use.

I think it is fair to say that EpicsCA has a more straightforward and
naive implementation than cothread, and provides a somewhat less
functional interface.  For example, EpicsCA forces explicit polling of
the CA library in user code.

So in summary, PythonCA is a bit too low level to be comfortable to use,
while both EpicsCA and Cothread wrap the low level details.  I think
Cothread is both more complete and smoother to use -- but having
designed it, I am biased here!

Implementation Dependencies

Cothread depends on ctypes, numpy and greenlet.  The numpy library is
used for both waveform support (not as well supported by the other
libraries, I believe) and to assist with data type conversion.  The
dependency on greenlet is the most awkward, but this is a very tiny
component that is easy enough to compile on all important targets.

EpicsCA depends on SWIG, and can use numpy if it is available.

PythonCA has been implemented largely as a C extension, so needs to be
compiled but has no dependency on SWIG.  There is a reference to numpy
in the source code, but I think it's an unused relic.

Threading Issues

The main important implementation issue where the three libraries differ
substantially is in the handling of threading.  In summary:

EpicsCA:
    Does not support any form of threading, and the EPICS CA library
    needs to be explicitly polled by user code.

PythonCA:
    Uses standard Python threads, and runs with preemptive callbacks
    enabled.

Cothread:
    Uses coroutines instead of standard threads and polls the EPICS CA
    library; however in this case the polling is concealed from user
code.

The argument between standard threads and cothreads is rather delicate,
and was covered in some detail in the posting referenced below:

 http://www.aps.anl.gov/epics/tech-talk/2009/msg00227.php

(See the section 'why "cothread"?')

In an early version of the cothread library we compared polled versus
preemptive callbacks and found that polling was quite a bit more
efficient when processing many camonitor updates, but this test should
probably be repeated.  Certainly cothread works fine with monitoring
several hundred PVs, each updating many times a second.

Cothread

Finally some comments on the cothread library.  Note that this doesn't
use SWIG, instead the ctypes library (now built into Python) allows C
functions to be called directly from Python without any compilation
required.  It's designed to be simple to use, but I must flag a point
that may need attention.

Note that cothreads are not classic (asynchronous) threads, instead
they're sometimes referred to as microthreads.  For most users the
distinction will be too subtle to be relevant, but there are a couple of
places where this will be visible.  Firstly, calling a blocking cothread
library function method (eg a Wait method) from another (classic) thread
will, in the current version of the cothread library, cause unexpected
problems.  I expect I can fix this, but the unwary programmer may have a
puzzling surprise.  The ThreadedEventQueue class is provided to help
with this issue.

Secondly, and but also fixable, there is an awkwardness in the
underlying implementation of coroutines provided by the greenlet
library: it currently works by swapping the entire active stack frame in
place (rather than by creating a separate stack for each cothread /
greenlet / coroutine / microthread, which is the more usual approach).
This doesn't disturb the operation of Python, but C extensions which
call into Python can be disrupted by this (this limits Qt4 integration,
for example).  I'd quite like to fix this by providing our own coroutine
library to replace greenlet, making it allocate a fresh stack for each
cothread.

I hope this is helpful.

    Michael Abbott

--
I apologise, I have no control over the stuff below.
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.

Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.

Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.

Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--
John Hammonds
Beamline Controls and Data Acquisition Group
APS Engineering Support Division

Argonne National Laboratory
[email protected] <mailto:[email protected]>
(630)252-5317

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System