Experimental Physics and
Industrial Control System

"Ernest L. Williams Jr." <[email protected]> · Mon, 21 Nov 2005 12:23:59 -0500

On Mon, 2005-11-21 at 08:29 -0600, Marty Kraimer wrote:
> Andrew,
> 
> Thanks for finding the missing isEpicsThread.
> 
> I fixed the memory leak.
> 
> I added two new fields to epicsThreadOSD
> isFifoScheduled
> isOnThreadList
> 
> isOnThreadList is set when the thread is added to the list. Thus 
> ellDelete is only called if this is true
> isFifoScheduled is used by epicsThreadSetPriority. If not true 
> epicsThreadSetPriority. silenly does not try to call posix routines to 
> set the priority. I do not know if this is necessary but it may be on 
> some posix implementations.
> 
> I have no way to test on my linux because it is old and does not seem to 
> support priority scheduling.
> 
> I will get Shifu to test on his linux systems which do.
> 
> Jeff and Earnest,
> 
> Can you try again?
Sure. Tests look good on the 32-bit linux machine for Fedora Core 3 and
4.  I am also able to exit the IOC cleanly.

========================================================================
[root@lion iocamd64]# cat /proc/version
Linux version 2.6.12-1.1381_FC3 ([email protected]) (gcc
version 3.4.4 20050721 (Red Hat 3.4.4-2)) #1 Fri Oct 21 03:46:55 EDT
2005


iocInit()
Starting iocInit
############################################################################
###  EPICS IOC CORE built on Nov 21 2005
###  EPICS R3.14.7 $$Name: R3-14-2_branch $$ $$Date: 2004/12/06 22:31:52
$$
############################################################################
iocInit: All initialization complete
## Start any sequence programs
#seq sncExample,"user=williamsHost"
epics> epicsThreadShowAll
            NAME     EPICS ID   PTHREAD ID   OSIPRI  OSSPRI  STATE
          _main_    0x9360128            0      0       0       OK
          errlog    0x9363658   3086908336     10      10       OK
          taskwd    0x9368f40   3086744496     10      10       OK
      timerQueue    0x9368b00   3086609328     70      69       OK
           cbLow    0x9366f18   3086343088     59      58       OK
        cbMedium    0x93670a8   3085814704     64      63       OK
          cbHigh    0x93a7300   3085286320     71      70       OK
        dbCaLink    0x93a7708   3084757936     50      50       OK
        scanOnce    0x93ac108   3084229552     70      69       OK
          scan10    0x93ac538   3083701168     60      59       OK
           scan5    0x93ac660   3083172784     61      60       OK
           scan2    0x93ac788   3082644400     62      61       OK
           scan1    0x93ac8b0   3082116016     63      62       OK
         scan0.5    0x93ac9d8   3081587632     64      63       OK
         scan0.2    0x93acb00   3081059248     65      64       OK
         scan0.1    0x93acc28   3080530864     66      65       OK
       CAC-event    0x93c82e0   3080002480     52      51       OK
         CAS-TCP    0x93ceee0   3079736240     16      16       OK
      CAS-beacon    0x93cf018   3079470000     14      14       OK
         CAS-UDP    0x93cf160   3079334832     12      12       OK
epics> dbl
williamsHost:aiExample
williamsHost:aiExample1
williamsHost:aiExample2
williamsHost:aiExample3
williamsHost:calcExample
williamsHost:calcExample1
williamsHost:calcExample2
williamsHost:calcExample3
williamsHost:compressExample
williamsHost:subExample
williamsHost:xxxExample
epics> casr(2)
Channel Access Server V4.11
Connected circuits:
TCP 160.91.232.168:53705(lion): User="williams", V4.11, 1 Channels,
Priority=50
        Task Id=0x93f17d0, Socket FD=3
        Secs since last send   0.71, Secs since last receive  10.17
        Unprocessed request bytes=0, Undelivered response bytes=0
        State=up
        416 bytes allocated
        williamsHost:aiExample(1rw)
UDP Server:
UDP 160.91.232.168:32784(): User="", V4.11, 0 Channels, Priority=0
        Task Id=0x93cf160, Socket FD=7
        Secs since last send  10.17, Secs since last receive  10.17
        Unprocessed request bytes=16, Undelivered response bytes=0
        State=up
        168 bytes allocated

There are currently 293748 bytes on the server's free list
6 client(s), 511 channel(s), 511 event(s) (monitors) 0 putNotify(s)
14 small buffers (16384 bytes ea), and 0 jumbo buffers (4000024 bytes
ea)
The server's resource id conversion table:
Bucket entries in use = 1 bytes in use = 16420
Bucket entries/hash id - mean = 0.000244 std dev = 0.015623 max = 1
The server's array size limit is 4000024 bytes max
Channel Access Address List
localhost.localdomain:5065
epics>

================================================================

Thanks,
Ernest


> 
> Marty
> 
> 
> Andrew Johnson wrote:
> 
> > Jeff Hill wrote:
> >
> >>
> >> One has to configure the build setting 
> >> "USE_POSIX_THREAD_PRIORITY_SCHEDULING
> >> = YES" on Linux in order to see these messages occurring whenever a 
> >> client
> >> connects.
> >
> >
> > Interesting - with that setting I can replicate this on our 
> > 2-processor linux server (saturn) but not on my 1-processor linux 
> > workstation (apsajnt).  I noticed that Jeff was running on an SMP 
> > kernel, and I assume his machine has more than one CPU:
> >
> >> Linux version 2.4.21-20.ELsmp ([email protected]) (gcc
> >> version 3.2.3 20030502 (Red Hat Linux 3.2.3-42)) #1 SMP Wed Aug 18 
> >> 20:46:40
> >> EDT 2004
> >
> >
> > My boxes look like this:
> >
> > saturn% cat /proc/version
> > Linux version 2.4.20-28.8smp ([email protected]) (gcc 
> > version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) #1 SMP Thu Dec 18 
> > 12:25:21 EST 2003
> >
> > apsajnt% cat /proc/version
> > Linux version 2.4.20-19.9 ([email protected]) (gcc 
> > version 3.2.2 20030222 (Red Hat Linux 3.2.2-5)) #1 Tue Jul 15 17:18:13 
> > EDT 2003
> >
> >
> > Looking at the code in libCom/osi/os/posix/osdThread.c I can now see 
> > the problem and explain why it's only happening on SMP systems.
> >
> > This code is from epicsThreadCreate():
> >
> >     pthreadInfo = init_threadInfo(name,priority,stackSize,funptr,parm);
> >     if(pthreadInfo==0) return 0;
> >     pthreadInfo->isEpicsThread = 1;
> >     setSchedulingPolicy(pthreadInfo,SCHED_FIFO);
> >     status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
> >                 start_routine,pthreadInfo);
> >     if(status==EPERM){
> >         pthreadInfo = init_threadInfo(name,priority,stackSize,funptr, 
> > parm);
> >         if(pthreadInfo==0) return 0;
> >         status = pthread_create(&pthreadInfo->tid,&pthreadInfo->attr,
> >                 start_routine,pthreadInfo);
> >     }
> >
> > The code creates a threadInfo object and calls setSchedulingPolicy(). 
> > If the subsequent pthread_create() call fails with an EPERM, it tries 
> > again without the setSchedulingPolicy().  There are two problems with 
> > this: a memory leak, since it never frees the original threadInfo 
> > object, and it doesn't set the isEpicsThread flag on the retry.  My 
> > conjecture is that on an SMP linux kernel pthread_create() does 
> > actually return EPERM if the user isn't root, whereas on my single 
> > processor (UP) kernel it just accepts (and probably ignores) the 
> > thread priority request without error.  I have tried running the IOC 
> > as root on my workstation, and epicsThreadShowAll lists all the 
> > threads with a zero for OSSPRI so it seems that the UP kernel 
> > completely ignores the thread priority. Unfortunately I don't have the 
> > ability to run the IOC as root on the SMP system, so I can't confirm 
> > my guess that way.
> >
> > I have committed a fix to the second problem (forgetting to set the 
> > isEpicsThread flag) which removes the messages Jeff was seeing, but 
> > the memory leak is harder to fix since the free_threadInfo() routine 
> > tries to remove the threadInfo object from a list which it hasn't been 
> > added to yet.  From the CVS history I see that Marty has already 
> > reworked this section of code a bit; I'm going to leave it for him to 
> > decide how to fix (there's a FIXME: comment at the relevent line).
> >
> > - Andrew
> 
>

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System