Experimental Physics and
Industrial Control System

"Ernest L. Williams Jr." <[email protected]> · Tue, 27 May 2003 15:16:57 -0400

On Tue, 2003-05-27 at 06:35, Hunt Steven wrote:
> Hi Ralph,
> 
> Yes, yes and yes  .....   as well as speeding up, building a release we need
> to much better manage the dependencies.  Particularly we need to track
> version numbers that make up a release, and make sure we can reproduce any
> of our standard configurations.  As you say, we are now able to control and
> track what is moved into production (but are not yet doing so for all
> systems), and what core release (VxWorks/Epics and Libraries) are used by an
> ioc. We have plans to take this further, more on what we have done at the
> epics meeting ....
We at the SNS are concerned about the same issues.  The environment
variable helped us mainly in the area of multi-lab collaboration.  When
one site checks in a module to the SNS CVS repository, the collaborating
partner lab can then CVS check-out, build and go.  We got fed-up with
changing hard-coded paths in the RELEASE files.  This works great for
rapid development between the labs.   I do agree with Ralph that we have
to be very careful for operations/production.  You could actually try
and CVS tag the environment file along with productions modules.

Also, don't forget to take advantage of the "ident" command so that you
can query run-time libraries, etc... for their version.  We are doing
this for vxWorks.  This allows us to see the run-time version of the
vxWorks kernel on every IOC.

We look forward to discussing more at the EPICS meeting. 

> 
> But on this particular issue (use the environment or not), we should be
> consistent, the build of base allows environment variables to be used, and
> so should building "Apps" such as the client ioc shell.
> 
> Some people don't drive safely, should we limit all cars to 40km/hr ?
> 
> Thanks Earnest for the patches to the old version, I will wait a day or two,
> in the hope that someone who understands perl will incorporate the changes
> into the new version, or perhaps I can just run this code as is for 3.14.1?

Yes, this will work for EPICS R3.14.1 as well.

> 
> Cheers
> Steve 
> 
> -----Original Message-----
> From: Ralph Lange [mailto:[email protected]]
> Sent: Tuesday, May 27, 2003 10:42 AM
> To: Steve Hunt
> Cc: EPICS Tech-Talk
> Subject: Re: convertRelease.pl
> 
> 
> Hi Steve,
> 
> just be careful: you might be throwing some air bags and security belts
> out of your car to make it run faster. Which works, of course.
> 
> 
> Some details:
> 
> Usually IOC applications are based on a combination of different support
> modules: EPICS base being one, downloaded or locally developed record,
> device and driver supports and other IOC applications being others. The
> number of these modules in an installation usually increases over the
> years: BESSY started with a few, currently we have 20.
> 
> The combination of these support modules and their release numbers is
> crucial. Some will depend on each other, many will depend on base. There
> is no obvious clean way to express these dependencies under a structure
> that uses a hierarchical make.
> 
> The problem is: If you just change and recompile one of the modules, you
> are not guaranteed that any of your IOC applications will still work.
> Some of them will, sure. Some will crash immediately, fine. Some others
> look like they would work, but they will just kill some IOCs under some
> circumstances some days later late at night on a weekend. (If I remember
> correctly, the JLab people have had some experience with that kind of
> problems.)
> 
> So: To help keeping things configurable _and_ stable, the complete set
> of module locations (which usually include the release numbers, at least
> with the standard directory name structure) is kept in one place: the
> RELEASE file(s) in the configure directory of a module.
> 
> This ensures that changes to the set of modules that an application is
> compiled against are made intentionally. It also ensures that any given
> application will be compiled against the same set of modules from the
> same locations, no matter who is running the "make" command or how many
> years have gone. (As long as you don't throw away the outdated versions
> of your modules.)
> 
> If you change this behaviour to be environment-controlled, you might
> very easily get into a situation where you are screwing up things with
> no way to get back to the working version.
> 
> Scenario:
> 
> You have a shared location for your module tree. Everything is in sync
> and fine. You keep your "standard" base release in an environment
> variable that is set by a global shell login script. Then - after your
> "standard" has been moved on three minor steps, since your installation
> procedures are so incredibly fast - one developer is changing one file
> in one of the modules and recompiles. The environment sets the new base
> location for that module, everything compiles fine, he/she doesn't get
> any warnings. Still: your complete set of modules is inconsistent now.
> So two weeks later one of your IOC hangs.
> 
> Since the paths for the modules (at least: base) are taken from the
> environment, you can look in the RELEASE files but these won't tell you
> which version of base the single modules where compiled against. Damn.
> You think: Must be some inconsistency. Better "make clean" and rebuild
> to get things straight. Right on. Suddenly (remember: your standard base
> release has changed) three other modules are just dumping error messages
> and don't compile anymore. Obviously they have never been compiled
> against the new standard base. Oops. You don't want to start getting
> into this, as you are not sure what to do and you have some pressure to
> get the IOC up again. So better get everything back to what it was
> before. But: Ouch - you don't know which release of base the other
> modules were compiled against before - this was taken from someone's
> environment and your own "make clean" just threw away all information
> who compiled the former versions and when.
> 
> Guess my point is clear ...
> 
> This is a worst case scenario, sure. But this is actually not too far
> from what happened to us some time ago. Setting everything from the
> environment you can easily manage to get yourself into a situation where
> you have to rely on the tape backup to get back to a running version.
> 
> 
> I'm not trying to scare anyone here - just trying to say that not
> setting all stuff from the environment was intentionally.
> 
> I know that the SLS is well aware of all this. I am sure that you have
> some other ways to keep the information which release number
> combinations are valid and tested and to keep track of your system
> progress: i.e. when what versions of which module combinations were
> moved to production by whom. Btw: I was always dreaming of a small mySQL
> database for this.
> 
> So actually the RELEASE file mechanism was more or less intended as an
> interface that complex installations could plug some database into where
> smaller projects could happily stay with the set of files.
> 
> Bottom line: I would be reluctant to make enviroment control the default
> behaviour as this would encourage people (I'm mainly thinking of smaller
> installations) to use only environment variables to save time and work
> and get themselves in trouble.
> 
> Sorry that this got lengthy ...
> 
> Cheers,
> Ralph
> 
> 
> >>>>> "Steve" == Steve Hunt <[email protected]> writes:
> 
>   > Hi all,
>   > In 3.14.1,  the script base/configure/tools/convertRelease.pl  seems to 
>   > expand configuration files (like configue/RELEASE), to create files to 
>   > be included in make for a particular application and architecture.   
>   > Unfortunately it does not take account of any environment variables, so 
>   > if you set EPICS_BASE in your environment it will be ignorred and the 
>   > value from your configuration files will be taken, or the build will 
>   > fail if they are not defined.
> 
>   > This is causing us a problem, as we don't want to manualy edit these 
>   > files every time we build epics in a different location (we are trying 
>   > to speed up, and generalize our  installation and build procedures).
> 
>   > Should this perl script be modified ? ( I can't do this as I can't even 
>   > spell pearl!).
> 
>   > Thanks in advance
>   > Steve Hunt

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System