Experimental Physics and
Industrial Control System

Hi Kay,

 

Yes, you are right.

 

My previous report that the problem did not occur on Centos7/RHEL7 was incorrect.  I was not seeing it there because I was actually running a binary for procServ that was built on a Fedora 15 system. 

 

On Fedora 15 /usr/include/asm-generic/socket.h contains the following:

 

#define SO_KEEPALIVE    9

#define SO_OOBINLINE    10

#define SO_NO_CHECK     11

#define SO_PRIORITY     12

#define SO_LINGER       13

#define SO_BSDCOMPAT    14

/* To add :#define SO_REUSEPORT 15 */

 

So SO_RESUSEPORT is not defined.

 

That file on Centos7 contains:

 

#define SO_OOBINLINE    10

#define SO_NO_CHECK     11

#define SO_PRIORITY     12

#define SO_LINGER       13

#define SO_BSDCOMPAT    14

#define SO_REUSEPORT    15

#ifndef SO_PASSCRED /* powerpc only differs in these */

#define SO_PASSCRED     16

#define SO_PEERCRED     17

#define SO_RCVLOWAT     18

#define SO_SNDLOWAT     19

#define SO_RCVTIMEO     20

 

So SO_REUSEPORT is defined.

 

When I then built procServ on Centos7 it has the same problems as Centos6, it creates multiple procServ processes listening on the same port.

 

With the patch I posted previously (removing the call to setsockopt completely) that problem does not occur.  However, it has another problem, which is that after killing a procServ with ^Q it cannot be restarted again for about 30 seconds, as Kay predicted.

 

I then made the changes shown in this patch, calling setsockopt with SO_RESUSEADDR rather than SO_REUSEPORT.

 

*********************************************

[root@ppu071 procServ-2.6.0]# diff -U3 acceptFactory.cc.orig acceptFactory.cc

--- acceptFactory.cc.orig       2016-07-14 16:41:59.985669326 -0500

+++ acceptFactory.cc    2016-07-14 16:42:18.288739774 -0500

@@ -14,10 +14,6 @@

#include <arpa/inet.h>

#include <string.h>

-#ifndef SO_REUSEPORT               // Linux doesn't know SO_REUSEPORT

-#define SO_REUSEPORT SO_REUSEADDR

-#endif

-

#include "procServ.h"

 class acceptItem : public connectionItem

@@ -61,7 +57,7 @@

     _fd = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);

     assert(_fd>0);

-    setsockopt(_fd, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof(optval));

+    setsockopt(_fd, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval));

     memset(&addr, 0, sizeof(addr));

     addr.sin_family = AF_INET;

*********************************************

 

This patch fixes both problems.  procServ will exit with an error if one tries to create a second instance listening on the same port, and it can be immediately restarted after quitting with ^Q.

 

Are there some platforms where SO_REUSEPORT should be used instead of SO_REUSEADDR?

 

Can we get a 2.6.1 release?

 

Thanks,

Mark

 

 

-----Original Message-----
From: Kasemir, Kay [mailto:[email protected]]
Sent: Thursday, July 14, 2016 4:02 PM
To: Mark Rivers; Eric Norum
Cc: [email protected]
Subject: Re: Problem with procServ on RHEL6

 

Hi:

 

>Interestingly at the beginning of acceptFactory.cc are these lines.

> 

>#ifndef SO_REUSEPORT               // Linux doesn't know SO_REUSEPORT

>#define SO_REUSEPORT SO_REUSEADDR

>#endif

> 

>So apparently at the time procServ was written Linux did not support SO_REUSEPORT.

> 

>I have 2 questions:

> 

> Was there a good reason for setting this option in procServ?

 

Just guessing here, but I always thought REUSEADDR is useful if a server is shut down without chance to properly clean up.

The socket would then be stuck in a wait state for some time, preventing you from restarting the server.

With REUSEADDR you can start the server back up right away.

 

Maybe REUSEPORT was used by accident, since it actually was a #define for REUSEADDR?

 

-Kay