Experimental Physics and
Industrial Control System

Matt Newville <[email protected]> · Wed, 20 Apr 2011 11:24:09 -0500

Hi Geoff, Luca,

Thanks for the message.  I find the posted traceback very confusing.
I'd like to see simple see example code that reproduces it.

Here's a few more details:

__on_connect() is the connection_callback for an epics.PV, called
indirectly by ca_create_channel() (C).  That is, creating an epics.PV
object calls ca.create_channel() (python) with PV.__on_connect as a
user-level connection callback.   ca.create_channel() (python) saves
this connection callback in a hashtable, and calls ca_create_channel()
(C) with a static, internal connection callback.  When that internal
connection callback is called, it looks at the hashtable and calls the
corresponding PV.__on_connect() method for the PV object.

The traceback says that the error originated from ca._onGetEvent()
(python), which the internally defined callback for monitored changes
-- the callback sent to ca_create_subscription (C).  That is as if a
monitored change called the connection callback instead of the monitor
callback (which would actually expect a 'count' keyword argument).

My interpretation would be that this (having the connection callback
called from the monitor handler) "cannot happen" (tm).   I've never
seen this behavior, though I do test disconnecting and reconnecting
PVs.    Hence, the request for example code.

Responding to a change in connection status by recreating a PV is not
necessary, and probably a bad idea, but I don't see immediately how it
would cause this problem.

Luca mentioned that this error happened with the PyQT GUI library, a
C++ library that probably makes heavy use of C++ threads.  For
wxPython (a similar situation), we definitely have to be careful to
prevent mixing GUI library threads with epics CA threads.  We have
pre-defined python decorators for mixing wx and CA code.    It's
possible that the PyQT library needs similar features, and it's
conceivable that traceback  messages from threads are interleaved, so
the ultimate problem might be related to this.   But without example
code, that's just speculation.

Finally, the pyepics module uses preemptive callbacks by default, so
it's actually not necessary to use ca.poll().

Cheers,

--Matt Newville

On Wed, Apr 20, 2011 at 10:43 AM, Geoff Savage <[email protected]> wrote:
> Hi Luca,
>
> I don't know anything about the internals of ca.py.  But I have read some
> python tracebacks over the years.
>
> This is telling you that the __on_connect() function does not understand the
> argument named "count" that is being passed to it.
> -------------------------------------------------------------------------------
> TypeError: __on_connect() got an unexpected keyword argument 'count'
> Traceback (most recent call last):
> File "_ctypes/callbacks.c", line 295, in 'calling callback function'
> File "/usr/local/lib/python2.6/dist-packages/epics/ca.py", line 420, in
> _onGetEvent
>  args.usr(value=value, **kwds)
> -------------------------------------------------------------------------------
> My guess is that you added a function that is called when the CA connection
> state changes and for this function you have included a count argument.  In
> the python interfaces to CA you almost never need to specify a count
> yourself.  The python code can determine the count with a query to the
> record.
>
> A connection callback should simply notify a user or send an alarm that the
> record can no longer be reached.  If you receive a disconnect callback you
> don't need to do anything except the default CA action. When the record is
> available again CA will automatically reconnect and you will receive a
> connect callback.
>
> The default CA action depends on your threading model.  We use single
> threaded applications with python GUIs with Tkinter.  In this case we use
> the after method of Tkinter widgets to call ca_poll() at 10Hz.  This allows
> CA to call the connection and monitor callbacks.
>
> CA is actually simpler to use than you think.  The connection handling has
> been coded at a level below the python interface.  At this point I should
> recognize the efforts of Jeff Hill and others to make the CA interface
> convenient to use.
>
> Geoff
>
>
> On 4/20/2011 2:21 AM, Luca Luisa wrote:
>>
>> Hi,
>> I tried to types of code:
>>
>> 1. in the first one I don't disconnect the PV when the Server
>> disconnects and I just wait an automatic reconnection.
>> 2. in the second I disconnect the PV, and re-create it again when the
>> server connects again.
>>
>> In both cases I get the same error.
>>
>> Luca
>>
>>
>>
>> On 04/19/2011 12:19 AM, Jeff Hill wrote:
>>>
>>> Hi Matt, Luca,
>>>
>>> Sorry about the delay responding - I was away on business at the end
>>> of last
>>> week.
>>>
>>>>> EPICS server, I get a lot of unknown errors in the terminal
>>>>> from where I launch my application, those are:
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>> TypeError: __on_connect() got an unexpected keyword argument 'count'
>>>>> Traceback (most recent call last):
>>>>> File "_ctypes/callbacks.c", line 295, in 'calling callback function'
>>>>> File "/usr/local/lib/python2.6/dist-packages/epics/ca.py",
>>>>> line 420, in _onGetEvent
>>>>> args.usr(value=value, **kwds)
>>>>> -----------------------------------------------------------------------
>>>>>
>>>
>>> Despite some similarities, I don’t believe that this particular message
>>> originates from the ca client library.
>>>
>>> Note also that if you do not unsubscribe when the circuit becomes
>>> unresponsive then the ca client library will auto-magically
>>> re-subscribe on
>>> your behalf should the channel need to be reconnected on a fresh TCP
>>> circuit.
>>>
>>> Jeff
>>> ______________________________________________________
>>> Jeffrey O. Hill Email [email protected]
>>> LANL MS H820 Voice 505 665 1831
>>> Los Alamos NM 87545 USA FAX 505 665 5107
>>>
>>> Message content: TSPA
>>>
>>> With sufficient thrust, pigs fly just fine. However, this is
>>> not necessarily a good idea. It is hard to be sure where they
>>> are going to land, and it could be dangerous sitting under them
>>> as they fly overhead. -- RFC 1925
>>>
>>>> -----Original Message-----
>>>> From: [email protected] [mailto:tech-talk-
>>>> [email protected]] On Behalf Of Matt Newville
>>>> Sent: Thursday, April 14, 2011 10:07 AM
>>>> To: [email protected]; EPICS Tech Talk
>>>> Subject: Re: ca.py - Throws error - workaround
>>>>
>>>> Hi Luca,
>>>>
>>>> I hope it's OK with you that I'm sending the message to Tech-Talk,
>>>> where there are many people with much better understanding of Channel
>>>> Access than I have.
>>>>
>>>> On Thu, Apr 14, 2011 at 1:58 AM, Luca Luisa<[email protected]> wrote:
>>>>>
>>>>> Hi,
>>>>> I'm working with PyEpics version 3.1.1.
>>>>> My QT Python application communicates with many EPICS servers where
>>>>
>>>> hundred
>>>>>
>>>>> of PVs are delivered.
>>>>>
>>>>> Our application has to deal with any EPICS server
>>>>> disconnection/re-connection that can happen during the application run.
>>>>>
>>>>> Each PV used by this QT Python application has a callback function
>>>>
>>>> added
>>>>>
>>>>> with the module "add_callback()".
>>>>>
>>>>> When I noticed an EPICS server disconnection, I clear the callbacks
>>>>
>>>> with the
>>>>>
>>>>> method "clear_callbacks()".
>>>>>
>>>>> When any EPICS server re-connect again, I try to add again another
>>>>
>>>> callback
>>>>>
>>>>> function to the same PV instance previously created, if fails (the
>>>>> add_callback returns None) I re-create the PV instance again and try to
>>>>
>>>> add
>>>>>
>>>>> the callback function.
>>>>>
>>>>> But in the cycle above described used to handle the dynamic connection
>>>>
>>>> to
>>>>>
>>>>> EPICS server, I get a lot of unknown errors in the terminal from where
>>>>
>>>> I
>>>>>
>>>>> launch my application, those are:
>>>>>
>>>>> -----------------------------------------------------------------------
>>>>
>>>> --------
>>>>>
>>>>> TypeError: __on_connect() got an unexpected keyword argument 'count'
>>>>> Traceback (most recent call last):
>>>>> File "_ctypes/callbacks.c", line 295, in 'calling callback function'
>>>>> File "/usr/local/lib/python2.6/dist-packages/epics/ca.py", line 420,
>>>>
>>>> in
>>>>>
>>>>> _onGetEvent
>>>>> args.usr(value=value, **kwds)
>>>>> -----------------------------------------------------------------------
>>>>
>>>> --------
>>>>>
>>>>> Our application is very complex so I cannot send you an example.
>>>>>
>>>>> In order to remove this error that full-fills the terminal window, I
>>>>
>>>> had to
>>>>>
>>>>> change the "ca.py" source with a try: except: statement.
>>>>> The diff is:
>>>>>
>>>>> 419,423c419,420
>>>>> < try:
>>>>> < if hasattr(args.usr, '__call__'):
>>>>> < args.usr(value=value, **kwds)
>>>>> < except:
>>>>> < """"""
>>>>> ---
>>>>>>
>>>>>> if hasattr(args.usr, '__call__'):
>>>>>> args.usr(value=value, **kwds)
>>>>>
>>>>> My solution is a pure workaround to make my application working and I
>>>>
>>>> got
>>>>>
>>>>> success but I would like to know if this is a known error or my way to
>>>>> handle connection/disconnection of a EPICS server is wrong.
>>>>>
>>>>> Many thanks
>>>>>
>>>>> Ing. Luca Luisa
>>>>
>>>> I'm reluctant to make that change, as it's the only place ca.py can
>>>> tell the programmer that there was an error in running the
>>>> user-supplied callback and this change silences that message.
>>>>
>>>> This message:
>>>>
>>>>> TypeError: __on_connect() got an unexpected keyword argument 'count'
>>>>> Traceback (most recent call last):
>>>>> File "_ctypes/callbacks.c", line 295, in 'calling callback function'
>>>>> File "/usr/local/lib/python2.6/dist-packages/epics/ca.py", line 420,
>>>>
>>>> in
>>>>>
>>>>> _onGetEvent
>>>>> args.usr(value=value, **kwds)
>>>>
>>>> says that PV.__on_connect() -- the internally defined connection
>>>> callback for a PV -- is being run as a change-in-value callback. I'm
>>>> not sure how that's even possible.
>>>>
>>>>> When I noticed an EPICS server disconnection, I clear the callbacks
>>>>
>>>> with the
>>>>>
>>>>> method "clear_callbacks()".
>>>>
>>>> I'm not sure this is necessary. Have you tried not doing this?
>>>> Obviously, you won't get a callback from a disconnected PV, and will
>>>> get a callback upon reconnection, but I would think that's not worse
>>>> than explicitly clearing and re-setting callbacks. I can verify that
>>>> it does works for me in simple test scripts to not clear "change
>>>> value" callbacks and restart the server IOC.
>>>>
>>>>> When any EPICS server re-connect again, I try to add again another
>>>>
>>>> callback
>>>>>
>>>>> function to the same PV instance previously created, if fails (the
>>>>> add_callback returns None) I re-create the PV instance again and try to
>>>>
>>>> add
>>>>>
>>>>> the callback function.
>>>>
>>>> For PV.add_callback() to return None, the PV has to be unconnected --
>>>> or at least that's the intention. So it's hard to understand what's
>>>> happening if you wait for reconnection to add a callback. I do think
>>>> it shouldn't be necessary to re-create the PV.
>>>>
>>>> I'm not sure whether this is related to using QT. A minimal example
>>>> would be really helpful.
>>>>
>>>> --Matt
>>>
>
>

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System