Argonne National Laboratory

Experimental Physics and
Industrial Control System

2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: Re: exporting module versions
From: Dirk Zimoch <dirk.zimoch@psi.ch>
To: Mark Rivers <rivers@cars.uchicago.edu>, "core-talk@aps.anl.gov" <core-talk@aps.anl.gov>
Date: Fri, 3 Nov 2017 12:08:29 +0100
On 02.11.2017 19:33, Mark Rivers wrote:
how can the library know whether the asynUser structure that was passed contains these fields or not?

It does not need to! I was talking about backward compatibility, not forward compatibility. So the question is would an old driver run with the new asyn version? I think the answer is yes.

The structure is created by asyn. Of course asyn knows about all current fields and the structure size.

A module using asyn calls createAsynUser() to get a structure pointer or some other asyn functions. It does not need to know how large the actual structure is because it uses only the pointer.

If the module had been compiled with an older version of asyn, the pointer now refers to something that is bigger than expected but the known part is still the same. The module will of course not use the unknown fields but the known fields are still fine.

The "only" thing asyn needs to ensure is not to fail if the module does not use these fields, for example if a device driver does not set them. So the fields need to be initialized to a "harmless" default value (like 0). This is indeed the case.

So there is no problem. The change is backward compatible.

But I can see that it is somehow difficult to give the promise that the ABI does not change -- in particular that the semantics did not change. For example if asyn would treat it as an error if a driver does not set the new fields, then we would have a problem. This is hard to validate from the source code and impossible to prove from the API alone. Only specific tests could show.

Dirk


On 02.11.2017 19:33, Mark Rivers wrote:
Hi Dirk,

* Add new fields only ever at the end of structure passed to API  functions by reference
(and then handle cases gracefully where the  fields don't exist).

Consider this change that was made 2 years ago to the asynUser structure in asynDriver.h.

corvette:asyn/asyn/asynDriver>git diff 325410dfaac1c0b000c32f515b8d9d96be15fefa ee6839f7c10bcc36454fec21ca0164c3579c9e55 asynDriver.h
diff --git a/asyn/asynDriver/asynDriver.h b/asyn/asynDriver/asynDriver.h
index 2d698ad..71ca083 100644
--- a/asyn/asynDriver/asynDriver.h
+++ b/asyn/asynDriver/asynDriver.h
@@ -56,16 +56,18 @@ typedef struct asynUser {
     char          *errorMessage;
     int            errorMessageSize;
     /* timeout must be set by the user */
...
+    int            auxStatus;     /* For auxillary status*/
+    int            alarmStatus;   /* Typically for EPICS record alarm status */
+    int            alarmSeverity; /* Typically for EPICS record alarm severity */
 }asynUser;

So I have followed your rule above, adding new fields to the end of the asyn user structure.  The implication of your message is that does not change the ABI, i.e. I don't need to make a new major version of asyn.

However, if code which was compiled against the old version of the header file calls this new library version how can the library know whether the asynUser structure that was passed contains these fields or not?

Mark

-----Original Message-----
From: core-talk-bounces@aps.anl.gov [mailto:core-talk-bounces@aps.anl.gov] On Behalf Of Dirk Zimoch
Sent: Thursday, November 02, 2017 7:43 AM
To: core-talk@aps.anl.gov
Subject: Re: exporting module versions


On 02.11.2017 11:18, Ralph Lange wrote:
Hi Dirk,

On Thu, Nov 2, 2017 at 10:23 AM, Dirk Zimoch <dirk.zimoch@psi.ch
<mailto:dirk.zimoch@psi.ch>> wrote:

    [...]  I also opt for binary backward compatibility, so that it is
    always possible to replace a dynamic library with a newer version
    without needing to re-build all programs. Forcing a program to link
    only with a very specific library version is, in my opinion, not
    very maintenance friendly.


As you seem to have experience with that: which tools / methodology do
you suggest to detect and track ABI changes in libraries, especially
libraries created from C++ sources?

Thanks,
~Ralph


I don't know any tools to support compatibility checks, but here is what
I try to do:

* Never remove API functions (declaring them depreciated is OK)
* Never change the signature of an existing (extern C) function (...in
an incompatible way. Signedness change is often OK. Adding const or
volatile where appropriate is also often OK. Changing 32 bit args (like
int) to potentially 64 bit args (e.g. size_t) is only OK if no 64 bit
was supported previously, but that is already ancient history.)
* Never change the semantics of an existing function (e.g. swap src and
dest parameters in some copy function)
* Never remove, re-order, or change size of the fields of a structure
that is used in an API.
* Add new fields only ever at the end of structure passed to API
functions by reference (and then handle cases gracefully where the
fields don't exist).
* Never remove or re-order virtual methods (the same for non-C++
function tables like in asyn).
* Expose as little as possible in the API. Not all functions are API
functions, not all structures/classes are used in the API. Not all
Macros are part of the API. Keep public and private header files
separate. Do not install private headers. This allows to change any non
API function, class, etc. at any time without breaking the API.
* Do not put private fields/methods in API classes. If private members
are needed, inherit from a API base class without private members. APIs
are not private.

As it is often not feasible to be so strictly backward compatible, I
suggest (any use in my software) the following rules:

* A version consists of 3 numbers: major.minor.patch
* Whenever a change is not binary backward compatible, the major number
increases.
* Whenever there are new features, the minor number increases.
* Whenever a bug is fixed without a new feature, the patch number
increases. (A bugfix may be incompatible in so far that the
incompatibility was the actual bug that has been fixed.)
* In linking use the major number in the file name to ensure no
incompatible version can be used.
* Do not use the minor number or patch number in linking in order to
allow upgrading the library.


See also how Linux (or GNU) does it: /bin/bash on my computer is linked
to libtinfo.so.5 which is a symbolic link to libtinfo.so.5.7. Note that
is is not linked to version 5.7 but only to version 5. This allows to
upgrade the library to 5.8 but not to 6.0 without having to rebuild the
executable.

Or: softIoc on my computer is linked to libstdc++.so.6 which is a
symbolic link to libstdc++.so.6.0.13.


I have no idea how to automatically check for backward compatibility
when releasing a new version. I can imagine checking function and
structure/class signatures automatically. But how to check for semantic
changes?


Dirk


Replies:
Re: exporting module versions Mark Rivers
References:
exporting module versions Michael Davidsaver
Re: exporting module versions Andrew Johnson
Re: exporting module versions Dirk Zimoch
Re: exporting module versions Ralph Lange
Re: exporting module versions Dirk Zimoch
RE: exporting module versions Mark Rivers

Navigate by Date:
Prev: RE: exporting module versions Mark Rivers
Next: Re: exporting module versions Mark Rivers
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: RE: exporting module versions Mark Rivers
Next: Re: exporting module versions Mark Rivers
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 03 Nov 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·