Experimental Physics and
Industrial Control System

Andrew Johnson <[email protected]> · Thu, 17 Nov 2005 17:16:36 -0600

Jeff Hill wrote:

One has to configure the build setting "USE_POSIX_THREAD_PRIORITY_SCHEDULING
= YES" on Linux in order to see these messages occurring whenever a client
connects.

Interesting - with that setting I can replicate this on our 2-processor

linux server (saturn) but not on my 1-processor linux workstation

(apsajnt). I noticed that Jeff was running on an SMP kernel, and I

assume his machine has more than one CPU:

Linux version 2.4.21-20.ELsmp ([email protected]) (gcc
version 3.2.3 20030502 (Red Hat Linux 3.2.3-42)) #1 SMP Wed Aug 18 20:46:40
EDT 2004

My boxes look like this:

saturn% cat /proc/version

Linux version 2.4.20-28.8smp ([email protected]) (gcc

version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) #1 SMP Thu Dec 18

12:25:21 EST 2003

apsajnt% cat /proc/version

Linux version 2.4.20-19.9 ([email protected]) (gcc

version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #1 Tue Jul 15 17:18:13

EDT 2003

Looking at the code in libCom/osi/os/posix/osdThread.c I can now see the

problem and explain why it's only happening on SMP systems.

This code is from epicsThreadCreate():

    pthreadInfo = init_threadInfo(name,priority,stackSize,funptr,parm);
    if(pthreadInfo==0) return 0;
    pthreadInfo->isEpicsThread = 1;
    setSchedulingPolicy(pthreadInfo,SCHED_FIFO);
    status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
                start_routine,pthreadInfo);
    if(status==EPERM){

 pthreadInfo = init_threadInfo(name,priority,stackSize,funptr,

parm);

        if(pthreadInfo==0) return 0;
        status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
                start_routine,pthreadInfo);
    }

The code creates a threadInfo object and calls setSchedulingPolicy(). If

the subsequent pthread_create() call fails with an EPERM, it tries again

without the setSchedulingPolicy(). There are two problems with this: a

memory leak, since it never frees the original threadInfo object, and it

doesn't set the isEpicsThread flag on the retry. My conjecture is that

on an SMP linux kernel pthread_create() does actually return EPERM if

the user isn't root, whereas on my single processor (UP) kernel it just

accepts (and probably ignores) the thread priority request without

error. I have tried running the IOC as root on my workstation, and

epicsThreadShowAll lists all the threads with a zero for OSSPRI so it

seems that the UP kernel completely ignores the thread priority.

Unfortunately I don't have the ability to run the IOC as root on the SMP

system, so I can't confirm my guess that way.

I have committed a fix to the second problem (forgetting to set the

isEpicsThread flag) which removes the messages Jeff was seeing, but the

memory leak is harder to fix since the free_threadInfo() routine tries

to remove the threadInfo object from a list which it hasn't been added

to yet. From the CVS history I see that Marty has already reworked this

section of code a bit; I'm going to leave it for him to decide how to

fix (there's a FIXME: comment at the relevent line).

- Andrew
--
* * Matt Santos / / Leo McGarry * * For a Brighter America * *

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System