EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  <20092010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  <20092010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Python Channel Access bindings
From: Michael Abbott <[email protected]>
To: EPICS Tech Talk <[email protected]>
Date: Thu, 5 Feb 2009 11:21:45 +0000 (GMT)
Channel Access bindings seems to be one of those areas which some people, 
including myself, just can't leave alone.  At Diamond we've been working 
on our own version, which I'd like to publicise now.

At http://controls.diamond.ac.uk/downloads/python/cothread you can 
download the latest version of our "cothread" library and see the detailed 
documentation online.  I'll explain the name and the logic in a moment -- 
the story is not simple!


The core functionality is, IMHO, nice and smooth -- in fact, it's so 
simple you'll ask why am I making such a fuss about it.  Here's some 
sample code:


import cothread
from cothread.catools import *  # Imports caget, caput, camonitor

print caget('TESTPV')           # Read from PV and print 
caput('TESTPV', 123)            # Write to PV

def monitor(value):
    print 'got', value

camonitor('TESTPV', monitor)
cothread.Sleep(10)              # Let monitor tick away for 10 seconds


The essential point is (and you'll have to read the documentation, online 
at http://controls.diamond.ac.uk/downloads/python/cothread/1-12/docs for 
details) that the easy case is simple (as shown above), but actually 
you'll find support for *all* relevant channel access features through 
this interface.

For example, if you want a timestamp, and you want the data in floating 
point that's easy enough:

x = caget('TESTPV', format=FORMAT_TIME, datatype=float)
print x, x.timestamp
# Ok, raw seconds is unfriendly.  Use this incanation for pretty print:
from datetime import datetime
print datetime.fromtimestamp(x.timestamp)


I've collapsed the myriad of possible channel access formats into three 
format choices (data only, data with timestamps and severity, data with 
all control fields), together with an orthogonal choice of datatype.  This 
seems to provide the cleanest user interface, and is fully compatible with 
what channel client access actually provides.

There are essential dependencies on the numpy and ctypes libraries -- all 
array data is returned using numpy arrays, and indeed numpy's built in 
conversion functionality is incredibly useful when converting between 
channel access dbr and Python data formats.  There is also a dependency on 
the greenlet Python library: this provides the underlying mechanism 
required to implement cothreads.  Finally, there is (just realised this) a 
hard-wired path to libca.so in cadef.py -- you'll need to edit this!


Now to the heart of the matter: why "cothread"?

A complete channel access binding library cannot dodge the problem of 
concurrency.  First and most important, camonitor events can arrive, in 
principle, at any time, and some mechanism needs to be provided to support 
this.  Also it is frequently a helpful optimisation to perform multiple 
channel access operations in parallel to reduce delays -- for example

xl = caget(['PV%d' % i for i in range(100)])

fetches a list of 100 values with all 100 connections and fetches running 
concurrently.  This can run 100 times as fast as doing the fetches 
sequentially.

So the obvious answer is to use threads ... but there are some snags.  
Whether the snags are large enough to justify the effort that's gone into 
this library is open to question, but the library exists and works quite 
well within its limitations.

The problems with using threads are:

1. Synchronisation.  This is the biggie, and everybody gets this wrong.  
We *still* don't have a programming model that prevents us from making 
embarassing synchronisation errors.

2. Python threads are ... slightly feeble.  The problem with Python 
threads is the "Global Interpreter Lock" -- this monster synchronisation 
point ensures that while Python is executing Python code there is actually 
only one thread active.  This means that C library calls can run 
concurrently, but Python code gains no benefit from multiple processors.
   The interpreter lock protects the Python interpreter from 
synchronisation errors, but doesn't help individual Python applications, 
as switching between Python threads occurs at random, as far as the 
application is concerned.

3. Threads are not cheap.  The caget([...]) call above spawns 100 
concurrent "cothreads" -- there's no way this would be affordable with 
real threads.  Of course there are other ways of doing this task, but the 
concurrent cothread solution is nice to write and well behaved, and the 
mechanism is now available for other applications.


So what *is* a cothread?  You could call it a "cooperative thread", and 
the underlying concept is called a "coroutine".  The primitive notion of 
coroutine is very basic and has been around for quite a long time.  The 
greenlet library which the cothread library depends on provides just one 
function to switch between coroutines: 
	x = task.switch(y)
This passes the value y to another task's .switch, where it is returned; 
when control is explicitly switched back to us we'll receive the value 
passed to that switch as the value assigned to x.

This primitive is very low level and actually very error prone to use in 
practice (keeping track of just which switches can occur when can become 
rather difficult).  The cothread library therefore wraps this so that 
cothreads behave very like ordinary threads -- except that control is only 
transferred between cothreads at well defined points.  This has involved 
the construction of a cothread scheduler which has become quite 
sophisticated.

The primitive cothread operations are: 
	Spawn() 	to create a new cothread
	Sleep(delay)	to suspend calling cothread for delay seconds
	event.Wait()	wait for event to become signalled
	event.Signal()	wake up another cothread waiting on an event
In this list only Sleep() and Wait() will result in control being lost to 
other cothreads (the complete list includes also Yield() and the catools 
functions, which call these primitives in their implementation).


The one big disadvantage of using cothreads is that if any cothread blocks 
waiting for input then the entire application will block.  It is possible 
to spawn a background thread for blocking I/O and communicate with it via 
the cothread.ThreadedEventQueue (only the main thread can run cothreads in 
the current implementation -- I was reluctant to add a thread local 
storage access to every scheduler reference, and there are some 
initialisation complexities).  Alternatively cothread.select or 
cothread.poll can be used to avoid blocking.


The cothread library also comes with Qt and readline bindings: this means 
that the interactive interpreter can be used to draw graphs, which can 
even update in the background while you type into the interpreter!  To 
enable the Qt bindings cothread.iqt() must be called, preferably before 
importing qt or PyQt4 -- note however that compromises have had to be made 
with Qt and there are some quirks, in particular modal windows are a 
disaster area.

See the documentation, see if the library works for you, let me know what 
you think.

Replies:
Problem with cothread-1-12 on MacOSX (Leopard) Juan Carlos Guzman

Navigate by Date:
Prev: Diamond Libera EPICS Driver Michael Abbott
Next: Re: EPICS Build without '-ansi' Andrew Johnson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  <20092010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: Diamond Libera EPICS Driver Michael Abbott
Next: Problem with cothread-1-12 on MacOSX (Leopard) Juan Carlos Guzman
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  <20092010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 31 Jan 2014 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·