Experimental Physics and
Industrial Control System

David Kelly <[email protected]> · Fri, 5 Dec 2003 13:44:29 +1100

Hi all,

I'm porting the EPICS R3.13.4 ioc base code to RedHawk Linux, and am seeking
some advice re several questions that I have.

(But first, some background info may be helpful.... I work for Concurrent
Computer Corp, who have a real-time version of Linux for x86 platforms,
called RedHawk Linux. This is not a real-time kernel which runs underneath
standard Linux, which is how RTLinux and L4-Linux work. Rather, it is an
open source enhanced version of the Linux kernel which includes all the
standard Linux stuff plus things like: true real-time priority scheduling,
fully pre-emptible kernel, a plethora of fast interprocess synchronization
and communication mechanisms, high resolution clocks/timers, extended memory
mapping capabilities, and the ability to shield processors from interrupts
and dedicate processes to specific processors in order to guarantee
deterministic system response, etc. It is currently based on the 2.4.21
kernel from kernel.org. It is installed by first installing Red Hat 8.0 on
the target system, and then installing the RedHawk kernel and real-time
libraries, etc. It is Red Hat 8.0 compatible. A propaganda page is here
http://www.ccur.com/isd_solutions_redhawklinux.asp if anyone wants more
info.)

The work I've done so far includes:

1) Create a "redhawk-x86" build environment.

2) Moved a few elements of the "posix" source files to the new "redhawk"
branch of the source tree and modify them to better take advantage of some
RedHawk features. The work done does not affect a lot of code, and it
remains largely based on the posix base. Most of the work is as follows,
though there some additional bits and pieces:
    - osdMutex: now uses RedHawk fast mutex mechanisms.
    - osdThread: has now enabled the existing
_POSIX_THREAD_PRIORITY_SCHEDULING code. This was not sufficient to enable
priority scheduling under RedHawk, so a small amount of extra code was
required (e.g. specifically enabling "SCHED_RR" [or SCHED-FIFO] on a
per-thread basis).
    - osdTime: Now uses CLOCK_REALTIME_HR for highest possible resolution. 

3) Compile the ioc base with all the changes and run the "epicsXXXXTest"
programs.

I would appreciate advice in two areas: some specific source code questions,
and then an initial much higher level question. (My biggest problem at the
moment is that I'm "looking at the wood, not the trees" - that is, I'm
hacking bits of low-level code without having much of an understanding of
how epics works as a whole. I have, however, read through the EPICS I/O
Controller Application Developer's Guide, some other EPICS training-type
docs from the web, and some whitepapers/technical reports which give an
overview of porting EPICS. Overall, these have been very helpful.....)

Firstly, I'd appreciate some help with code-specific queries:

a) The posix/osdThread.c routines uses "pthread_mutex_XXXX()" functions for
synchronization, rather than the platform specific mutexes in osdMutex.c. Is
there any reason for this ? (I've changed osdThread.c to use the mutexes in
osdMutex.c and everything seems OK so far...)

b) The posix/osdTime.cpp file appears to contain current references to
"gmtime_r()" and "localtime_r()" functions, which I gather are specific to
VxWorks. These do not appear to be resolved anywhere in a build under Linux
(or RedHawk). Is this correct, or am I missing something ?

c) I don't yet understand the way the thread priorities are calculated in
posix/osdThread.c, when _POSIX_THREAD_PRIORITY_SCHEDULING is enabled. Can
someone please explain to me the effect that the getOssPriorityValue()
function is intended to have ?

d) Under RedHawk, if a process desires to run with a real-time priority
(i.e. use a real-time scheduling policy of SCHED_RR or SCHED_FIFO, where the
real-time priorities range from 99 down to 1 and are all therefore higher
than the standard Linux SCHED_OTHER timesharing priority of zero), then the
process must be owned by, or run in the context of, the root user. I could
do this by logging on as root and MAKEing the code as root. However, I
prefer to MAKE the project in the context of my normal user login, which
means that I then need to "chown root XXXX" and "chmod a+s XXXX" all the
created binary executables so that they run as the root user. I'd like to be
able to automate the "chown" and "chmod" as part of the build (MAKE)
process, but I haven't yet figured out how to modify the build environment
to do this. Can anyone suggest how to do this, or at least where I should be
looking ?

e) What other areas of the code should I be looking at, in order to improve
the real-time performance (in a Linux-type environment) ?

Now for the "trees"-type question....

i) I'm doing this work on my laptop computer, with some further testing on a
multi-processor system in our office. In this restricted environment, what's
the simplest way to test that the resultant EPICS "application" runs
correctly ? In the same vein, what's the simplest way to run a
performance/response-time test of the EPICS/ioc code ? (I'd like to run some
performance/response measurements under standard Linux and then again under
RedHawk. Am thinking of doing something like that described in the "Porting
EPICS to L4-Linux Based System" whitepaper, where a thread grabs the CPU
counter, kicks an I/O module into interrupting, again grabs the CPU counter,
and then measures the difference in the counter values....)

Thanks for any and all advice.

Kind regards,

David Kelly

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System