Experimental Physics and
Industrial Control System

Matt Newville <[email protected]> · Fri, 2 Oct 2009 17:13:34 -0500

Hi,

Sorry for the delay in responding -- I wasn't receiving tech-talk mail.

> > For example, I don't see significant advantages to having all the C CA
> > types (DBR_SHORT, DBR_INT, DBR_LONG, etc) exposed to Python at all, as
> > Python does not distinguish these types, and the difference never
> > needs to be exposed to the Python programmer.
>
> It /is/ a difference to the server whether you request DBR_SHORT, DBR_INT,
> or DBR_LONG. These will trigger different conversion routines, and thus
> might give different results.

Of course.  But Python does not have separate native datatypes for
int, short, long and does not distinguish unsigned and signed integer
types (there are extensions that can make this distinction, mostly
used to pack data for other C libraries).   So, the Python programmer
should never be forced to make the distinction, or even have to know
that it exists.
> > The DBR_ type for a channel is an implementation detail that is
> > important in C, but not in Python.
>
> If it is not important in Python, then why should it be important in C?

Because they are different languages with different needs and
requirements.   Type casting, pointer arithmetic, and memory
management are also somehow more important in C than Python.

> > One should be able to create a channel/PV in Python without
> > knowing its underlying type.
>
> Yes, but field type is different from request type!

Of course, but when (in Python) would you want something that is a
native DOUBLE as a CTRL_ENUM?  Never.  If you want it a native DOUBLE
as an integer,
      print  int(caget(MyDoublePVname))
is easier than
      print  caget(MyDoublePVname, type=ca.DBR_SHORT)

You may very well want a DOUBLE as a CTRL_DOUBLE (and this is and must
be supported by all the Python interfaces available).
It's also true that the C types don't map one-to-one onto Python
types, so asking for a native short as a long makes very little sense.

In short,  getting the native type and allowing a CTRL (and TIME) type
are sufficient.

> You /don't/ want to use the native (field) type for all queries, for
> instance overlong string fields might actually be implemented in the
> database as a DBF_CHAR array. Requesting the native type will be wrong, in
> this case. OTOH, a DBF_CHAR array might also be meant as a vector of 8-bit
> numbers. The client program must know and choose the correct interpretation
> by chosing the correct request type.

Sure, in fact in my own extension, I keep the "char" variation of the
value around all the time... it's quite useful to have a double
formatted to a string with the correct precision and to have the enum
Integer and String value simultaneously.

> I think it is a very good idea to map the /complete/ C API to a low-level python CA lib.

Well, I'd be willing to use a complete API that was well debugged, but
I'm not sure I agree that it is a very good idea to map the complete
API.  I am definitely a little skeptical that a complete mapping of
the 3.14.11 API is actually important.....

Like, I cannot understand why DBR_STS_* and DBR_GR_* are ever needed
in Python:  If the extra expense of a DBR_CTRL_* versus DBR_GR_* is a
problem, then Python is definitely the wrong choice.

In addition, mixing threads well between C and Python is well known to
be hard and error-prone.    A "complete" API would probably need to
allow "ca_create_context(ca_enable_preemptive_context)". I'm not sure
that even make sense with a language with its own VM.  How is this
*supposed* work in such a case? Is it exposed in other extensions
languages? Perl? Matlab???  I looked, but couldn't find any positive
answers.   Even if it could be done, is it a good idea to have this?

For what it's worth, My own extension and cothreads do not enable
preemptive callbacks, and neither has "native" threads -- my own
simply does not have them at all.
Thus, if I understand correctly, most of the available Python
interfaces do not provide a complete mapping of the API.

Which other wrappers provide a "complete" API?     Do any Perl or
Matlab or other extensions expose threads and preemptive callbacks?

Andrew Johnson wrote:
> I should add that the request type also affects the amount of data passing
> over the wire; if you request a large image with 8-bit pixels as a DBF_LONG or
> worse a DBF_DOUBLE array, you're going to be wasting network bandwidth and CPU
> time on the IOC converting all those individual 8-bit values into larger
> quantities.  Much better to do that conversion on the client.

I agree that it's best for the interface code to ask for the data
efficiently and so (in this case), get the request type right.   I'll
that I'm saying is that, when you get the data Python, the issue is
not SHORT, LONG, or DOUBLE.  In fact, it is the much more interesting
and useful question of simple list of numbers, Numpy 2-D array
(already for mathematical manipulation) or a Python Image (all ready
to have a color mapping applied).  It's a different language, with
different uses, and expecting the API to be "complete, and just like
C" focuses attention on the wrong questions.

--Matt Newville <newville at cars.uchicago.edu>

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System