This document pre-dates the addition of generalTime support to EPICS Base.
This document describes the behaviour of the IOC clock synchronization software for R3.13 and R3.14 IOCs that use the APS Event system. It was written for use at APS, but may be useful for other labs too. — Andrew Johnson
| Release | OS | Event H/W | TSconfigure Settings | IOC Time Class |
| 3.13.10 | VxWorks | No | None | Soft slave |
| 3.13.10 | VxWorks | No | Master=No, Inhibit=No | Soft slave |
| 3.13.10 | VxWorks | No | Master=No, Inhibit=Yes | Soft slave |
| 3.13.10 | VxWorks | No | Master=Yes, Inhibit=No | 3.13 Soft master |
| 3.13.10 | VxWorks | No | Master=Yes, Inhibit=Yes | 3.13 Soft master |
| 3.13.10 | VxWorks | Yes | None | 3.13 Event slave |
| 3.13.10 | VxWorks | Yes | Master=No, Inhibit=No | 3.13 Event slave |
| 3.13.10 | VxWorks | Yes | Master=No, Inhibit=Yes | Soft slave |
| 3.13.10 | VxWorks | Yes | Master=Yes, Inhibit=No | 3.13 Event master |
| 3.13.10 | VxWorks | Yes | Master=Yes, Inhibit=Yes | 3.13 Soft master |
| 3.14.x | Workstation | No | None | OS Clock |
| 3.14.x | VxWorks | No | None | IocClock slave |
| 3.14.x | VxWorks | No | Master=No, Inhibit=No | Soft slave |
| 3.14.x | VxWorks | No | Master=No, Inhibit=Yes | Soft slave |
| 3.14.x | VxWorks | No | Master=Yes, Inhibit=No | 3.14 Soft master |
| 3.14.x | VxWorks | No | Master=Yes, Inhibit=Yes | 3.14 Soft master |
| 3.14.x | VxWorks | Yes | None | IocClock slave |
| 3.14.x | VxWorks | Yes | Master=No, Inhibit=No | 3.14 Event slave |
| 3.14.x | VxWorks | Yes | Master=No, Inhibit=Yes | Soft slave |
| 3.14.x | VxWorks | Yes | Master=Yes, Inhibit=No | 3.14 Event master |
| 3.14.x | VxWorks | Yes | Master=Yes, Inhibit=Yes | 3.14 Soft master |
| 3.14.9 | RTEMS | No | None | RTEMS time |
| 3.14.9 | RTEMS | No | Master=No, Inhibit=No | RTEMS time |
| 3.14.9 | RTEMS | No | Master=No, Inhibit=Yes | RTEMS time |
| 3.14.9 | RTEMS | No | Master=Yes, Inhibit=No | RTEMS time |
| 3.14.9 | RTEMS | No | Master=Yes, Inhibit=Yes | RTEMS time |
| 3.14.9 | RTEMS | Yes | None | RTEMS time |
| 3.14.9 | RTEMS | Yes | Master=No, Inhibit=No | 3.14 Event slave |
| 3.14.9 | RTEMS | Yes | Master=No, Inhibit=Yes | RTEMS time |
| 3.14.9 | RTEMS | Yes | Master=Yes, Inhibit=No | 3.14 Event master |
| 3.14.9 | RTEMS | Yes | Master=Yes, Inhibit=Yes | RTEMS time |
| 3.14.10 | RTEMS | No | None | osiNTP slave |
| 3.14.10 | RTEMS | No | Master=No, Inhibit=No | Soft slave |
| 3.14.10 | RTEMS | No | Master=No, Inhibit=Yes | Soft slave |
| 3.14.10 | RTEMS | No | Master=Yes, Inhibit=No | 3.14 Soft master |
| 3.14.10 | RTEMS | No | Master=Yes, Inhibit=Yes | 3.14 Soft master |
| 3.14.10 | RTEMS | Yes | None | osiNTP slave |
| 3.14.10 | RTEMS | Yes | Master=No, Inhibit=No | 3.14 Event slave |
| 3.14.10 | RTEMS | Yes | Master=No, Inhibit=Yes | Soft slave |
| 3.14.10 | RTEMS | Yes | Master=Yes, Inhibit=No | 3.14 Event master |
| 3.14.10 | RTEMS | Yes | Master=Yes, Inhibit=Yes | 3.14 Soft master |
| IOC Time Class | Initial Time Sources | Sync Time Sources | Sync Frequency |
| Soft slave | NTP/Unix/Master | Master/NTP | sync_rate (10 seconds) |
| A ‘Soft slave’ uses the network to obtain time data, synchronizing to a Master IOC if one is available or to an NTP server if not. The EPICS_TS_NTP_INET variable provides the initial time server address; on vxWorks it will try to obtain the time from its boot host if that variable is not set. The Unix UDP time protocol does not return fractional seconds, which is why it is is only used for the initial time when an NTP server is unavailable — Unix UDP time cannot be used for synchronization. The Master IOC it talks to may be either a ‘Soft master’ or an ‘Event master’ — both run the appropriate synchronization server task which responds to UDP broadcast messages from slaves. The master IOC is discovered by sending a broadcast UDP message on port 18323; a reply is expected within 250ms, and the responding host is noted as the master IOC. To keep synchronized, a slave polls its master IOC with a UDP message to port 18323 every sync_rate seconds and updates its clock with the response time, adjusted by half of the round-trip message delay. If at any time the Master IOC stops responding, the slave will poll its NTP server instead until a master comes back on-line, which it detects by occasionally sending out UDP broadcasts on port 18323 as before. Small time corrections (up to 1second/second) will be applied gradually over the sync period, but as of latest version of apsEvent (R3-3) there is currently no provision to prevent the clock time from going backwards when a larger correction is needed. | |||
| 3.13 Soft master | NTP/Unix | NTP | On demand |
| 3.14 Soft master | NTP/Unix | None | N/A |
| A ‘Soft master’ uses the network to obtain initial time data, and then provides synchronization services to any ‘Soft slave’ IOCs on the same subnet. The EPICS_TS_NTP_INET variable provides the initial time server address; on vxWorks it will try to obtain the time from its boot host if that variable is not set. The Unix UDP time protocol does not return fractional seconds, which is why it is is only used for the initial time when an NTP server is unavailable — Unix UDP time cannot be used for synchronization. On initialization, the master starts a network server that listens for UDP broadcast messages on port 18323 and replies to timestamp requests from slave IOCs with the master's current time. The version of drvTS in Base R3.13 gives a Soft master the ability to synchronize with an NTP server on demand by calling the TSsetClockFromUnix() routine. Unfortunately the R3.14 version in the apsEvent support application appears to have removed this capability from the soft master. Small time corrections (up to 1second/second) will be applied gradually over the sync period, but as of latest version of apsEvent (R3-3) there is currently no provision to prevent the clock time from going backwards when a larger correction is needed, which will then flow down to all the slave IOCs. | |||
| 3.13 Event slave | NTP/Unix/Master | Event Master | Master's sync_rate (10 s) |
| 3.14 Event slave | NTP/Unix/Master | Event Master | Master's sync_rate (10 s) |
| An ‘Event slave’ contains a hardware event receiver for time synchronization with its Event Master, and also listens for network packets that provide additional information about the hardware events.. The slave initially uses the network to obtain the time from a time server just like the soft slaves do, but it then starts waiting for hardware events and listening on the network for UDP packets on port 18322 from the Master which contain the time that the last timing synchronization hardware event that was sent out. A 3.14 Event slave can read synchronization reporting threshold data from the longer network packets sent out by a 3.14 Event master. This additional information is ignored by the older 3.13 Event slaves. Small time corrections (up to 1second/second) will be applied gradually over the sync period, but as of latest version of apsEvent (R3-3) there is currently no provision to prevent the clock time from going backwards when a larger correction is needed. | |||
| 3.13 Event master | NTP/Unix | NTP | On demand |
| 3.14 Event master | NTP/Unix | NTP | On demand |
| An ‘Event master’ contains a hardware event generator, and sends out time synchronization pulses and UDP network packets to the ‘Event slave’ IOCs on its subnet. The EPICS_TS_NTP_INET variable provides the initial time server address; on vxWorks it will try to obtain the time from its boot host if that variable is not set. The Unix UDP time protocol does not return fractional seconds, which is why it is is only used for the initial time when an NTP server is unavailable — Unix UDP time cannot be used for synchronization. On initialization, the master starts a synchronization task that generates hardware event pulses every sync_rate seconds and immediately afterwards broadcasts a UDP packet on port 18322 which provides the time at the moment when that event occurred for the slave to synchronize with. A 3.14 Event master generates a longer network packet than a 3.13 master, with additional information on reporting thresholds for the clients to use. This extension was designed to be both forwards and backwards compatible, so 3.13 slaves can be used with a 3.14 master and vice versa. An Event master can be made to synchronize itself with its NTP server by calling the TSsetClockFromUnix() routine. Small time corrections (up to 1second/second) are applied gradually over the sync period, but as of latest version of apsEvent (R3-3) there is currently no provision to prevent the clock time from going backwards when a larger correction is needed, which will then flow down to all the slave IOCs. | |||
| OS Clock | OS (usually NTP) | OS (usually NTP) | N/A |
| Soft IOCs rely on the OS kernel and any associated clock daemons to maintain time synchronization. The Solaris and Linux systems at APS usually run an NTP daemon. In the .8 subnet the main NTP server is phoebus, which ultimately syncs with delta (a Stratum 1 GPS-connected TYMServ) and gamma (a Stratum 2 IRIG-B connected TYMServ). In the .2 subnet the two machines boreas and zephyr are the main NTP servers which themselves sync with delta and gamma as in the .8 subnet. | |||
| IocClock slave | NTP | NTP | 10 seconds |
| An ‘IocClock slave’ uses the network to obtain time data, synchronizing to an NTP server. During IOC operation the time is synchronized with this server every 10 seconds. A mechanism is included to prevent the clock time from going backwards due to a synchronization event. On vxWorks the EPICS_TS_NTP_INET variable provides the initial time server address, but the boot host is used if that variable is not set. On RTEMS the NTP server's address is provided as part of the OS network configuration. | |||
| RTEMS time | BSP (NTP/RTC) | None | N/A |
| IN R3.14.9 an RTEMS IOC that is not configured to use a hardware event system relies on the OS Kernel to keep time. This gets its initial time from an NTP server (configured with the same mechanism as it uses for network settings, which may be from DHCP or Flash) or from any on-board Real-Time Clock chip. After the initial time setting, the RTEMS OS clock free runs — it has no external synchronization capability. | |||
| osiNTP slave | NTP | NTP | 60 seconds |
| An ‘osiNTP slave’ uses the network to obtain time data, synchronizing to an NTP server. During IOC operation the time is synchronized with this server every 60 seconds. A mechanism is included in the General Time subsystem to prevent the clock time from going backwards. On vxWorks the EPICS_TS_NTP_INET variable provides the initial NTP time server address, but the boot host is used if that variable is not set. On RTEMS the NTP server's address is provided as part of the OS network configuration. | |||