EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  <20162017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  <20162017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: set-user-id root and EPICS 3.15
From: Ralph Lange <[email protected]>
To: EPICS Tech-Talk <[email protected]>
Date: Thu, 28 Jan 2016 15:43:26 +0100
Well,

As memory locking is active, you may actually be running against a per-process memory limit on your machine.

This locking of memory happens when POSIX priority scheduling is active.
It was suggested as an automatic improvement for real-time IOCs, recognizable by running under priority scheduling. real-time IOCs will always want to run memlock'd, as page faults and swapping is something that doesn't play well with real-time requirements.

However, this is the second time that we hear of trouble because people are running client applications (i.e. CA clients) in real-time context. In those cases, the memory locking might not be intended nor useful. There is no way for the OS abstraction layer (where the memlock happens) to know if the caller is an IOC type or a client type application.
We already began to reconsider the automatic activation feature.

Meanwhile:
If you patch at the place that Mark pointed out to deactivate the code (make the first line '#if 0'), you should be fine.

Sorry for the inconvenience,
~Ralph


On 28/01/2016 15:24, Mark Rivers wrote:
Hi Gerry,

I don't know what the problem is, but my first guess is to look at the difference between the posix code in EPICS 3.14 and 3.15.  In particular osdProcess.c and osdThread.c.

diff -U3 /usr/local/epics/base-3.14.12.5/src/libCom/osi/os/posix/osdThread.c /usr/local/epics/base-3.15.2/src/libCom/osi/os/posix/

Here is a difference that looks looks like it could be relevant, for example:

+
+#if defined(_POSIX_MEMLOCK) && _POSIX_MEMLOCK > 0
+    if(errVerbose)  {
+        fprintf(stderr, "LRT: min priority: %d max priority %d\n",
+            pcommonAttr->minPriority, pcommonAttr->maxPriority);
+    }
+    if (pcommonAttr->maxPriority > pcommonAttr->minPriority) {
+        status = mlockall(MCL_CURRENT | MCL_FUTURE);
+        if(status) {
+            fprintf(stderr, "Unable to lock the virtual address space using mlockall\n");
+        } else {
+            fprintf(stderr,"Successfully locked memory using mlockAll\n");
+        }
+    }
+#endif
+

Mark


________________________________________
From: [email protected] [[email protected]] on behalf of Gerry Swislow [[email protected]]
Sent: Wednesday, January 27, 2016 8:52 PM
To: [email protected]
Subject: set-user-id root and EPICS 3.15

Hi Tech-Talkers,

I'm having a problem with EPICS 3.15 in the "spec" set-user-id-root executable.

Why do I need a set-user-id-root binary?  Many sites use spec on Linux platforms with PC board motor controllers, counter/timers, GPIB interfaces, etc.  For many years now, spec has eschewed kernel drivers in favor of user-level support.  The user-level support works on all Linux distributions over all iterations of the Linux kernel with no effort needed by the users. (Try that with kernel drivers!)

User level support requires access to I/O ports or on-board memory, which requires super-user privilege.  Thus, in most installations, spec is installed as a set-user-id root program.  The privilege escalation is immediately reset to that of the real user first thing when spec starts, and is only reactivated for the needed system calls, along these lines:

         if (setreuid(-1, 0) == 0) {
                 iopl(3);
                 setreuid(-1, getuid());
         }

There is no issue with users having root access while running spec with respect to permissions to do anything else that would not normally be permitted.

The problem is that in such an environment, certain library or system calls fail in a strange way, but only when spec is linked with EPICS 3.15.  There is no issue with EPICS 3.14.  When run as a normal user, the following set-user-id-root test program fails with the messages:

  Successfully locked memory using mlockAll
  pthread_create error Resource temporarily unavailable

Of course, if run as root, there is no error.  Using the command "sudo strace -u gerry ./a.out", I see that the error is associated with a mmap() call:

mmap(NULL, 2101248, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = -1 EAGAIN (Resource temporarily unavailable)

In spec, the errors occur with shmat(), getpwuid(), which uses mmap(), and malloc().  spec fails for good when malloc() fails.

I should also point out that the EPICS call isn't even needed.  Just the act of linking with the EPICS 3.15 libraries breaks the code.

To be specific, I used EPICS 3.15.3 and EPICS 3.14.12.  I tested on CentOS 7.2 and Linux Mint 17.2, both 64 bits.  And I've spent all day trying to figure this one out.

Test code and a build script follow.  Does anybody have a clue?

Thanks,

Gerry

-------------------------------------
/* file foo.c */#include <cadef.h>

int
main() {
         char    *p;

         setreuid(-1, getuid());
         ca_context_create(ca_disable_preemptive_callback);
}

-------------------------------------
#!/bin/bash

BASE=/usr/local/epics/base-3.14.12
BASE=/usr/local/epics/base-3.15.3

cc -I$BASE/include/os/Linux -I$BASE/include  -I$BASE/include/compiler/gcc \
    foo.c -L$BASE/lib/linux-x86_64 -lca -lCom

sudo chown root a.out
sudo chmod u+s a.out
./a.out

-------------------------------------


-------------------------------------
     Gerry Swislow
     Certified Scientific Software
     PO Box 390640
     Cambridge, MA  02139-0007

   phone:  (617) 576-1610
      fax:  (617) 497-4242
     email:  [email protected]
        web:  http://www.certif.com





Replies:
Re: set-user-id root and EPICS 3.15 Andrew Johnson
References:
set-user-id root and EPICS 3.15 Gerry Swislow
RE: set-user-id root and EPICS 3.15 Mark Rivers

Navigate by Date:
Prev: RE: set-user-id root and EPICS 3.15 Mark Rivers
Next: Re: set-user-id root and EPICS 3.15 Gerry Swislow
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  <20162017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: set-user-id root and EPICS 3.15 Mark Rivers
Next: Re: set-user-id root and EPICS 3.15 Andrew Johnson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  <20162017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 15 Jul 2016 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·