Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: RTEMS+zoneset seems to be broken
From: Ricardo Cardenes <rcardenes@gemini.edu>
To: tech-talk@aps.anl.gov
Date: Fri, 27 Oct 2017 14:45:11 -1000

Hi all,

A year ago, Matt Rippa reached to the list[1] because he was having trouble setting the time zone at runtime on EPICS+RTEMS.

To remind you the problem: the firmware for some of our VME boards' firmware is PPCBug bootloeader, and for some others it is MOTLoad.

In the MOTLoad-based CPUs there's not much of a problem: we could just set epics-tz in MOTLoad the environment, and that will be picked-up by RTEMS when reading info from the NVRAM, defining in turn the environment variable TZ. This is not possible (or at least it is not implemented) for the PPCBug boards, meaning that we have to set the time zone in some other way. epicsEnvSet() would let us set the appropriate variables from the startup script, but at this point in the execution, there was no way to call tzset() to make RTEMS/newlib change the system-wide timezone.

Michael Davidsaver proposed a patch[2] introducing "zoneset" as an iocsh function, precisely to be able to do that (set TZ, then call tzet()).

It worked for us, but we've noticed an inconsistency: sometimes we get an unexpected behaviour, that we won't see in the next reboot.

Reading rtems_init.c:Init (the beginning of _main_ thread) I've realized that most probably this is due to a race condition. The startup sequence (simplified) goes like this:

  1) Read config from NVRAM
  2) Set the priority for _main_
  3) Init the console
  4) Start the network and initialize the remote filesystem
  5) Use the BSP supplied time of the day, if available
  6) Set TZ (if needed) and call tzset
  7) osdTimeRegister() (creates the NTPTimeSync and ClockTimeSync threads)
  8) Execute the startup script
  9) Call main()

Now, up until 7), there are only 3 threads in the system (_main_, ImsgDaemon, errlog), but this step creates two new threads to synchronize time sources.

My conclusion is that there is a race condition where the call to zoneset() (which would be the first in the startup script), does sometimes execute before the sync threads get to set anything in the system; but sometimes the sync threads manage to do some significant work before the time zone is changed to the desired one, screwing up the internal view that the EPICS/RTEMS/newlib system has of the time (we use also use Bancomm bc635 as an additional time source in some of those systems, which may, or may not, be further compounding the issue).

I'm not 100% sure of the exact effects of such a race condition, as I'm still going through the code trying to figure out all the implications.

Our fix so far has been to use instead the epicsRtemsInitPostSetBootConfigFromNVRAM hook, which is not as flexible (needs to be compiled in, where zoneset goes in the startup script), but is 100% stable.

Now... Is there any reasonable solution to this?

Regards,
Ricardo


Replies:
Re: RTEMS+zoneset seems to be broken Michael Davidsaver

Navigate by Date:
Prev: Re: Implementation of a generic rest module extracted from ADEiger Bruno Martins
Next: How to define a global variable and global array in python script for a widget of Control system studio (CSS) ? lzf neu
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Using the stream device @mismatch handler as an if conditional Paduan Donadio, Marcio
Next: Re: RTEMS+zoneset seems to be broken Michael Davidsaver
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 29 Oct 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·