Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: Re: RTEMS+zoneset seems to be broken
From: Michael Davidsaver <mdavidsaver@gmail.com>
To: Ricardo Cardenes <rcardenes@gemini.edu>, tech-talk@aps.anl.gov
Date: Sun, 29 Oct 2017 23:01:17 -0500
Hi Ricardo,

Forgive me for being dense, but what is the unexpected behavior that you
see?  That the zoneset() has no effect?  That the effect come and then
goes?  That the time has some random offset?  What if you run zoneset()
again?

From a quick test, with RTEMS 4.9.6 I can use zoneset() to change the
output of generalTimeReport() after initialization.

> qemu> generalTimeReport 5
> Backwards time errors prevented 7 times.
>
> Current Time Providers:
>     "EVR", priority = 50
>         Current Time not available
>     "NTP", priority = 100
>         Current Time is 2017-10-29 21:56:33.953748.
>     "OS Clock", priority = 999
>         Current Time is 2017-10-29 21:56:33.964115.
>
> Event Time Providers:
>     "EVR", priority = 50
>
> qemu> zoneset HST10HST10
> qemu> generalTimeReport 5
> Backwards time errors prevented 7 times.
>
> Current Time Providers:
>     "EVR", priority = 50
>         Current Time not available
>     "NTP", priority = 100
>         Current Time is 2017-10-29 17:56:47.362454.
>     "OS Clock", priority = 999
>         Current Time is 2017-10-29 17:56:47.372182.
>
> Event Time Providers:
>     "EVR", priority = 50



I can find two candidates for wierdness.  One in newlib which I don't
think is relevant as it's only used by eg. strftime(), mktime(), and
similar.  And one in RTEMS relating to the NTP client which may be if
you use DHCP/BOOTP for address assignment.

The first is __gettzinfo() which gives access to the global variable
where zone info is stored (there is an associated lock).  This is set by
tzset() and used only when time is decomposed.

https://github.com/epicsdeb/rtems-gcc-newlib/search?utf8=%E2%9C%93&q=__gettzinfo&type=

The second is rtems_bsdnet_timeoffset which appears to be picked up from
a dhcp/bootp server.  It's used when shifting NTP server time to posix
time (I'm not clear on why this is desirable).  If you use dynamic
addressing you might check if your server returns the 'time-offset' (#2)
option.

https://github.com/epicsdeb/rtems/search?utf8=%E2%9C%93&q=rtems_bsdnet_timeoffset&type=


On 10/27/2017 07:45 PM, Ricardo Cardenes wrote:
> 
> Hi all,
> 
> A year ago, Matt Rippa reached to the list[1] because he was having
> trouble setting the time zone at runtime on EPICS+RTEMS.
> 
> To remind you the problem: the firmware for some of our VME boards'
> firmware is PPCBug bootloeader, and for some others it is MOTLoad.
> 
> In the MOTLoad-based CPUs there's not much of a problem: we could just
> set epics-tz in MOTLoad the environment, and that will be picked-up by
> RTEMS when reading info from the NVRAM, defining in turn the environment
> variable TZ. This is not possible (or at least it is not implemented)
> for the PPCBug boards, meaning that we have to set the time zone in some
> other way. epicsEnvSet() would let us set the appropriate variables from
> the startup script, but at this point in the execution, there was no way
> to call tzset() to make RTEMS/newlib change the system-wide timezone.
> 
> Michael Davidsaver proposed a patch[2] introducing "zoneset" as an iocsh
> function, precisely to be able to do that (set TZ, then call tzet()).
> 
> It worked for us, but we've noticed an inconsistency: sometimes we get
> an unexpected behaviour, that we won't see in the next reboot.
> 
> Reading rtems_init.c:Init (the beginning of _main_ thread) I've realized
> that most probably this is due to a race condition. The startup sequence
> (simplified) goes like this:
> 
>   1) Read config from NVRAM
>   2) Set the priority for _main_
>   3) Init the console
>   4) Start the network and initialize the remote filesystem
>   5) Use the BSP supplied time of the day, if available
>   6) Set TZ (if needed) and call tzset
>   7) osdTimeRegister() (*creates the NTPTimeSync and ClockTimeSync threads*)
>   8) Execute the startup script
>   9) Call main()
> 
> Now, up until 7), there are only 3 threads in the system (_main_,
> ImsgDaemon, errlog), but this step creates two new threads to
> synchronize time sources.
> 
> My conclusion is that there is a race condition where the call to
> zoneset() (which would be the first in the startup script), does
> sometimes execute /before/ the sync threads get to set anything in the
> system; but sometimes the sync threads manage to do some significant
> work before the time zone is changed to the desired one, screwing up the
> internal view that the EPICS/RTEMS/newlib system has of the time (we use
> also use Bancomm bc635 as an additional time source in some of those
> systems, which may, or may not, be further compounding the issue).
> 
> I'm not 100% sure of the exact effects of such a race condition, as I'm
> still going through the code trying to figure out all the implications.
> 
> Our fix so far has been to use instead
> the epicsRtemsInitPostSetBootConfigFromNVRAM hook, which is not as
> flexible (needs to be compiled in, where zoneset goes in the startup
> script), but is 100% stable.
> 
> Now... Is there any reasonable solution to this?
> 
> Regards,
> Ricardo
> 
> [1] http://www.aps.anl.gov/epics/tech-talk/2016/msg00720.php
> [2] http://www.aps.anl.gov/epics/tech-talk/2016/msg00733.php


Replies:
Re: RTEMS+zoneset seems to be broken Matt Rippa
References:
RTEMS+zoneset seems to be broken Ricardo Cardenes

Navigate by Date:
Prev: Re: How to define a global variable and global array in python script for a widget of Control system studio (CSS) ? Anton Derbenev
Next: Optimum Switch Over Time of Master and Slave IOC Michaelsmith2017
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: RTEMS+zoneset seems to be broken Ricardo Cardenes
Next: Re: RTEMS+zoneset seems to be broken Matt Rippa
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 30 Oct 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·