Experimental Physics and
Industrial Control System

Torsten Bögershausen <[email protected]> · Thu, 2 Nov 2017 15:45:18 +0100

On 02/11/17 15:26, Michael Davidsaver wrote:

Can we agree on the distinction between *documenting*, *detecting*, and
*avoiding* ABI changes?

The recipe I give is a simple way to *document* ABI changes.  This is
something which I think every module with an API should be doing.

As for *avoiding* ABI changes.  IMO this is really only practical when
the language, compiler, and underlying libraries are designed with this
in mind.  The Linux world has never focused on this (though glibc is
getting better).  And C++ makes maintaining a stable ABI quite difficult.

That is dependent on the definition of "Linux world".
This are my experiences:

- The kernel internally is refactored all the time
- The kernel API/ABI is very strict in binary compatibily
  (And int stays an int, and is not changed to size_t or ssize_t)
- The user space library libc is very strict (not to confuse with glibc)
- The C++ library make life harder (but it should be possible to
  compile under RHEL 6 and run it under Centos 7 for example)

In the EPICS world, it is not always clear what is a pure bug-fix and
what is a feature or improvement. I have seen too many bug fixes that
fix one thing for one application and (unintentionally) break another one -
So I would agree with Mark here:
What does it cost and what do we gain ?

In between these two is the question of *detecting* ABI changes.  I
haven't had sufficient motivation to do this.  This is why my recipe
defaults to using the module version as SONAME.  This is encoding the
assumption that every release contains ABI changes (and developers
working with VCS checkouts are on there own).

On 11/02/2017 08:32 AM, Mark Rivers wrote:

Hi Dirk,

I agree that ABI compatibility is desirable in theory.  It is essential for things like operating system libraries.  But for EPICS applications a cost/benefit analysis is needed.  I would argue that for the limited number of libraries needed for CA clients (libCom and ca) having ABI compatibility is good, we can replace those libraries without having to relink.

But for IOCs is it worth it?  The main benefit for having ABI compatibility is eliminating the time needed to rebuild the IOC application.  However, on my Linux system rebuilding everything except EPICS base and V4 (i.e. synApps, seq, asyn, stream, areaDetector and over 100 IOCs) takes 113 seconds which is pretty quick.  Not having ABI compatibility does mean that I need to maintain the source code trees so that this rebuild is possible.  But I better do that anyway in case I need to change the configuration of an IOC or add a new one.  The cost of ABI compatibility is to constrain the developer from making the types of internal changes we currently do.  We would also need to make major releases much more frequently which may intimidate or confuse sites who only care about API compatibility.

I would be curious to know how many sites currently try to simply replace libraries (e.g. libCom, ca, asyn, seq) for their IOCs, or would do so if it was better supported, compared to rebuilding from scratch?

In my experience the larger problems are in the lack of ability to deploy binaries across various OS versions.  I can't build an application on my Centos 7 development system and deploy it on RHEL 6.  (I can build an application on Windows 10 and deploy it on XP.  Microsoft does a better job here.)  The C++ compiler runtime ABIs are typically not backward compatible from one release of a compiler to the next.  This is certainly true of Visual Studio, and it is true for certain versions of g++ as well.

Mark

________________________________
From: [email protected] <[email protected]> on behalf of Dirk Zimoch <[email protected]>
Sent: Thursday, November 2, 2017 7:42 AM
To: [email protected]
Subject: Re: exporting module versions

On 02.11.2017 11:18, Ralph Lange wrote:

Hi Dirk,

On Thu, Nov 2, 2017 at 10:23 AM, Dirk Zimoch <[email protected]
<mailto:[email protected]>> wrote:

     [...]  I also opt for binary backward compatibility, so that it is
     always possible to replace a dynamic library with a newer version
     without needing to re-build all programs. Forcing a program to link
     only with a very specific library version is, in my opinion, not
     very maintenance friendly.

As you seem to have experience with that: which tools / methodology do
you suggest to detect and track ABI changes in libraries, especially
libraries created from C++ sources?

Thanks,
~Ralph

I don't know any tools to support compatibility checks, but here is what
I try to do:

* Never remove API functions (declaring them depreciated is OK)
* Never change the signature of an existing (extern C) function (...in
an incompatible way. Signedness change is often OK. Adding const or
volatile where appropriate is also often OK. Changing 32 bit args (like
int) to potentially 64 bit args (e.g. size_t) is only OK if no 64 bit
was supported previously, but that is already ancient history.)
* Never change the semantics of an existing function (e.g. swap src and
dest parameters in some copy function)
* Never remove, re-order, or change size of the fields of a structure
that is used in an API.
* Add new fields only ever at the end of structure passed to API
functions by reference (and then handle cases gracefully where the
fields don't exist).
* Never remove or re-order virtual methods (the same for non-C++
function tables like in asyn).
* Expose as little as possible in the API. Not all functions are API
functions, not all structures/classes are used in the API. Not all
Macros are part of the API. Keep public and private header files
separate. Do not install private headers. This allows to change any non
API function, class, etc. at any time without breaking the API.
* Do not put private fields/methods in API classes. If private members
are needed, inherit from a API base class without private members. APIs
are not private.

As it is often not feasible to be so strictly backward compatible, I
suggest (any use in my software) the following rules:

* A version consists of 3 numbers: major.minor.patch
* Whenever a change is not binary backward compatible, the major number
increases.
* Whenever there are new features, the minor number increases.
* Whenever a bug is fixed without a new feature, the patch number
increases. (A bugfix may be incompatible in so far that the
incompatibility was the actual bug that has been fixed.)
* In linking use the major number in the file name to ensure no
incompatible version can be used.
* Do not use the minor number or patch number in linking in order to
allow upgrading the library.

See also how Linux (or GNU) does it: /bin/bash on my computer is linked
to libtinfo.so.5 which is a symbolic link to libtinfo.so.5.7. Note that
is is not linked to version 5.7 but only to version 5. This allows to
upgrade the library to 5.8 but not to 6.0 without having to rebuild the
executable.

Or: softIoc on my computer is linked to libstdc++.so.6 which is a
symbolic link to libstdc++.so.6.0.13.

I have no idea how to automatically check for backward compatibility
when releasing a new version. I can imagine checking function and
structure/class signatures automatically. But how to check for semantic
changes?

Dirk

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System