Experimental Physics and
Industrial Control System

"Jeff Hill" <[email protected]> · Fri, 18 Nov 2005 08:22:22 -0700

> 
> Interesting - with that setting I can replicate this on our 2-processor
> linux server (saturn) but not on my 1-processor linux workstation
> (apsajnt).  I noticed that Jeff was running on an SMP kernel, and I
> assume his machine has more than one CPU:

Yes, that was our fast server - with an SMP kernel and it does have multiple
CPUs installed on the MB.

> I'm going to leave it for him to decide

Ok, thanks for having a look.

Jeff

> 
> > Linux version 2.4.21-20.ELsmp ([email protected]) (gcc
> > version 3.2.3 20030502 (Red Hat Linux 3.2.3-42)) #1 SMP Wed Aug 18
> 20:46:40
> > EDT 2004
> 
> My boxes look like this:
> 
> saturn% cat /proc/version
> Linux version 2.4.20-28.8smp ([email protected]) (gcc
> version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) #1 SMP Thu Dec 18
> 12:25:21 EST 2003
> 
> apsajnt% cat /proc/version
> Linux version 2.4.20-19.9 ([email protected]) (gcc
> version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #1 Tue Jul 15 17:18:13
> EDT 2003
> 
> 
> Looking at the code in libCom/osi/os/posix/osdThread.c I can now see the
> problem and explain why it's only happening on SMP systems.
> 
> This code is from epicsThreadCreate():
> 
>      pthreadInfo = init_threadInfo(name,priority,stackSize,funptr,parm);
>      if(pthreadInfo==0) return 0;
>      pthreadInfo->isEpicsThread = 1;
>      setSchedulingPolicy(pthreadInfo,SCHED_FIFO);
>      status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
>                  start_routine,pthreadInfo);
>      if(status==EPERM){
>          pthreadInfo = init_threadInfo(name,priority,stackSize,funptr,
> parm);
>          if(pthreadInfo==0) return 0;
>          status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
>                  start_routine,pthreadInfo);
>      }
> 
> The code creates a threadInfo object and calls setSchedulingPolicy(). If
> the subsequent pthread_create() call fails with an EPERM, it tries again
> without the setSchedulingPolicy().  There are two problems with this: a
> memory leak, since it never frees the original threadInfo object, and it
> doesn't set the isEpicsThread flag on the retry.  My conjecture is that
> on an SMP linux kernel pthread_create() does actually return EPERM if
> the user isn't root, whereas on my single processor (UP) kernel it just
> accepts (and probably ignores) the thread priority request without
> error.  I have tried running the IOC as root on my workstation, and
> epicsThreadShowAll lists all the threads with a zero for OSSPRI so it
> seems that the UP kernel completely ignores the thread priority.
> Unfortunately I don't have the ability to run the IOC as root on the SMP
> system, so I can't confirm my guess that way.
> 
> I have committed a fix to the second problem (forgetting to set the
> isEpicsThread flag) which removes the messages Jeff was seeing, but the
> memory leak is harder to fix since the free_threadInfo() routine tries
> to remove the threadInfo object from a list which it hasn't been added
> to yet.  From the CVS history I see that Marty has already reworked this
> section of code a bit; I'm going to leave it for him to decide how to
> fix (there's a FIXME: comment at the relevent line).
> 
> - Andrew
> --
> * * Matt Santos / / Leo McGarry * * For a Brighter America * *

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System