Subject: |
Re: Philosophy regarding use of open source libraries for EPICS |
From: |
Bruce Hill <[email protected]> |
To: |
<[email protected]> |
Date: |
Wed, 16 Nov 2016 15:30:10 -0800 |
Hi Rod,
We've faced the same problem here at SLAC. Our environment has a wide variety of host machines, including ioc hosts,
build hosts, operator consoles, and analysis stations. Originally these were RHEL5, most now are RHEL6, with RHEL7
being introduced. Each of these has a different libc, requiring different builds for each. Relying on yum repo
installation of third party packages wasn't working as the available package versions were often way out of date.
Most of these hosts are diskless, booting a standard image, so to avoid needing to install all the third party packages
needed for both EPICS development and analysis on each host, we build and install these to networked file systems.
To handle this, we created a python based system we call package manager. It consists of python scripts that know how
to build and install typical c/c++ unix packages from tarballs using ./configure, make, make install, and also how to
build and install python packages using setup.py build and setup.py install. For each package type, there's also a
dependency file to specify version specific package dependencies, and a short python file to handle quirks of each
package type. For an easy package the new python code is about 10 lines of boilerplate, but we've also handled builds
which use qmake, autogen, cmake, etc. We can also customize configure arguments, build arguments, add missing
headers, customize installs etc.
In practice it means for each new package type, an engineer will download the tarball, tweak the package specific python
to get it to build, and check it in. Then we can build that package, and usually subsequent versions of the package,
for different host targets and at multiple build sites for different areas at SLAC.
We can now build over 180 different third party packages, including the most recent V4 EPICS-CPP 4.6.0.
There's also a release feature, where a release file specifies a version specific set of packages. When the release
is built, it builds all the required packages including their dependencies, then creates bin, lib, include, and
site-packages directories w/ soft links to all the relevant files from each package and it's dependents. Env scripts
add bin to PATH, lib to LD_LIBRARY_PATH and PYTHONPATH, site-packages to PYTHONPATH, etc.
This has been particularly useful for python applications, allowing us to create applications which launch w/ their own
custom python environment, thus avoiding potential conflicts w/ other python packages.
For EPICS builds, we typically strip out and/or disable builds of vendor packages, modifying CONFIG_SITE to specify
versions and locations for hdf5, jpeg, gif, aravis, ffmpeg, and other packages.
In regards to your question about how far down to go re dependent packages, we usually stop for common packages which
are readily available in the standard Redhat images.
Regards,
- Bruce
On 11/16/2016 07:43 AM, Rod Nussbaumer wrote:
All:
I would like to get peoples' opinions about the right way to deal with open source third party libraries that get
bound to EPICS applications, particularly on Linux, but wherever else it comes into play. An example case would be
libgif, which can be used by EDM to display GIF images in EDM screens. The Area Detector package seems to make
significant use of such libraries for image processing. So far, the standard procedure here has been to install the
libraries from binary repositories supported by OS vendors such as Redhat, Debian, etc. We do this on development
hosts and the matching production host architectures. However, this leaves us vulnerable to incompatible changes that
may be introduced due to routine maintenance of the hosts, and silently changes the deployed EPICS software with no
knowledge of the effect, or testing that it works as expected. The alternative would be to build from sources, all
libraries that go into an EPICS application, and install the lib binaries in a way that they get used in lieue of the
system-installed packages. Does anyone do that? Am I being paranoid? The obvious question is how far would you take
this approach, since literally everything binds to libc, libm, and some other standard libraries.
Thanks in advance for your opinions.
Rod Nussbaumer
TRIUMF
Vancouver, Canada
--
Bruce Hill
Member Technical Staff
SLAC National Accelerator Lab
2575 Sand Hill Road M/S 10
Menlo Park, CA 94025
- References:
- Philosophy regarding use of open source libraries for EPICS Rod Nussbaumer
- Navigate by Date:
- Prev:
Re: Philosophy regarding use of open source libraries for EPICS J. Lewis Muir
- Next:
Re: Philosophy regarding use of open source libraries for EPICS Konrad, Martin
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
<2016>
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Philosophy regarding use of open source libraries for EPICS J. Lewis Muir
- Next:
Re: Philosophy regarding use of open source libraries for EPICS Konrad, Martin
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
<2016>
2017
2018
2019
2020
2021
2022
2023
2024
|