Experimental Physics and
Industrial Control System

Andrew.

I agree with you that fast EPICS scan rates are probably not
going to be reliable but I wanted to make the point that the
reason is not the underlying OS.

A RTOS (including linux/RT_PREEMPT) on a modern machine
can reasonably schedule multi-kHz tasks with worst-case
latency/jitter in the order of some low tens of microseconds.

If you have a hard real-time job then you can do this in a
dedicated high-priority thread and then e.g., feed cooked
and timestamped data into EPICS records.

I have used this approach for years now (and yes, I have
stuff running at several kHz) in multiple applications
and it works as expected.

- Till

On 01/08/2014 09:37 AM, Andrew Johnson wrote:

Hi Till,

The periodic scan threads use epicsEventWaitWithTimeout() to wait for
their next tick — we need to be able to wake up those threads on demand
so the IOC doesn't have to spend up to 10 seconds (or whatever is the
longest scan period) waiting for those threads to shut down when the
user types exit.

On the Posix architectures the wait occurs in pthread_cond_timedwait(),
on RTEMS in rtems_semaphore_obtain(), on VxWorks in semTake(), and on
Windows in WaitForSingleObject(). The Posix API provides nanosecond
resolution, and even the ability to choose which clock to use although
we don't set that ourselves and that ability may not be supported on all
architectures. Note that our Posix code runs on MacOS and Solaris as
well as Linux.

For RTEMS and VxWorks the wait time is expressed in system clock ticks.
Increasing the tick rate to use a finer time resolution results in more
CPU time spent handling the tick interrupt, as Dirk has shown; that is a
trade-off the user can make though. On Windows the delay time is given
in milliseconds so the maximum scan rate there would be 1KHz, assuming
that the OS could cope.

Personally I think that trying to support scan rates faster than about
1KHz at most is never going to be reliable and probably isn't worth
spending much time on, but I'm always open to code contributions that
prove me wrong (as long as they're sufficiently portable).

- Andrew

On 01/08/2014 10:36 AM, Till Straumann wrote:

It should be noted that with RT_PREEMPT the real-time threads
all execute in *user-mode*. They can run side-by-side with other
low-priority stuff in the same process and still get scheduled as
needed, preempting any other (lower-priority) thread or process
with low latency. Hence it is not necessary to play special tricks
with putting high-priority code into kernel modules etc.

- Till


On 01/08/2014 08:20 AM, Till Straumann wrote:

On 01/07/2014 02:19 PM, Andrew Johnson wrote:

Periodic scans are quite complicated, since they rely on the
underlying operating system to implement a delay timer and reschedule
the scan thread after the specific delay has elapsed. Most OSs don't
provide high accuracy delay scheduling, so I don't expect we will ever
be able to run periodic scan threads at 10KHz. - Andrew

As a matter of fact I don't think that 10kHz periodic scans are a big
problem for something like RTEMS
or linux (with RT_PREEMPT) on a reasonably fast CPU.

I just used a simple 'clock_nanosleep' based loop (based on code by
windriver, modified by myself)
which does something like

while ( 1 ) {
       /* get current time */
         clock_gettime( CLOCK_MONOTONIC, &wakeup );
       /* compute wakeup time 100us from now   */
         timespec_add_100us( &wakeup );
       /* sleep until wakeup time      */
         clock_nanosleep( CLOCK_MONOTONIC, TIMER_ABSTIME, &wakeup, NULL );
       /* see when we actually woke up */
         clock_gettime( CLOCK_MONOTONIC, &now );
       /* missed = now - wakeup        */
         timespec_diff( &missed, &now, &wakeup );

       compute_statistics( &missed );
}

for a bunch of threads (using the SCHED_FIFO scheduler).

On my laptop (vanilla kernel, no RT_PREEMPT) I observe worst-case
'missed' deadline of several milli-seconds
but even on that system the running average is between 5us..50us.

On a kernel patched with RT_PREEMPT I get a worst-case missed deadline
by ~10us (average around 3-4us)
after running for a few minutes (on a not heavily loaded intel core
i7-2655LE, 2.2 GHz).

I would expect RTEMS/vxWorks to even perform a bit better.

- Till

References:

Re: Increasing scan rate to 10 kHz Andrew Johnson

Re: Increasing scan rate to 10 kHz Till Straumann

Re: Increasing scan rate to 10 kHz Till Straumann

Re: Increasing scan rate to 10 kHz Andrew Johnson

Navigate by Date:

Prev: Re: Increasing scan rate to 10 kHz Andrew Johnson

Next: RE: VLANS designing，Geographical vs functional? Emmanuel Mayssat

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <2014>  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: Re: Increasing scan rate to 10 kHz Andrew Johnson

Next: Re: Increasing scan rate to 10 kHz Dirk Zimoch

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <2014>  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024