CODEWITHSUNDEEP
c++linuxsocketsboostboost-asio
JanCG
JanCG

Reputation: 105

Avoid socket inheritance when starting Linux service from C++ application

I have a Linux service (daemon) that has multiple-threads and uses boost io_service listening on a TCP socket. When I receive a certain message on that socket I want to start another service with e.g. /etc/init.d/corosync start.

The problem is that after starting the service, when I quit my own service, the other service has inherited the sockets from my own service and it remains in a strange status where I cannot stop it the usual way.

Before exiting my process "MonitorSipServer" the open socket shows like this:

netstat -anop |grep 144
tcp        0      0 0.0.0.0:20144           0.0.0.0:*               LISTEN          4480/MonitorSipServ off (0.00/0/0)
tcp        0      0 140.0.24.181:20144      140.0.101.75:47036      ESTABLISHED 4480/MonitorSipServ off (0.00/0/0)

After exiting my process "MonitorSipServer" the open socket shows like this:

netstat -anop |grep 144
tcp        0      0 0.0.0.0:20144           0.0.0.0:*               LISTEN      4502/corosync    off (0.00/0/0)
tcp        0      0 140.0.24.181:20144      140.0.101.75:47036      ESTABLISHED 4502/corosync    off (0.00/0/0)

I have already tried with system, popen and with fork + execv or execve with null environment. It's always the same result or worse. My last hope was the Linux setsid command, but it has not worked either.

Any help would be appreciated. Regards, Jan

Upvotes: 6

Views: 2805

Answers (3)

Tanner Sansbury
Tanner Sansbury

Reputation: 51931

Boost.Asio supports the fork() system call:

  • It requires the program to prepare and notify the io_service of the fork with io_service::notify_fork():

    io_service_.notify_fork(boost::asio::io_service::fork_prepare);
if (fork() == 0)
{
  io_service_.notify_fork(boost::asio::io_service::fork_child);
  ...
}
else
{
  io_service_.notify_fork(boost::asio::io_service::fork_parent);
  ...
}

    Failing to use this pattern results in unspecified behavior. For some configurations, the parent will fail to receive event notifications, as they are consumed by the child.

  • It is the program's responsibility to handle any file descriptors accessible through Boost.Asio's public API. For example, if the parent has an open acceptor, then the child would need to explicitly invoke acceptor::close() before spawning its own children processes or replacing the process image. The fork support documentation states:

    Note that any file descriptors accessible via Boost.Asio's public API (e.g. the descriptors underlying basic_socket<>, posix::stream_descriptor, etc.) are not altered during a fork. It is the program's responsibility to manage these as required.

  • Boost.Asio's fork support is not safe for multi-threaded processes. For multi-threaded processes, fork() states that the child may only invoke async-signal-safe operations between fork() and one of the exec() functions. io_service::notify_fork() does not make this guarantee, and the current implementation invokes non async-signal-safe operations.

With that said, I have seen applications not observe undesirable behavior when using Boost.Asio's fork() support in a multi-threaded process. However, if one wishes to meet the requirements of both fork() and Boost.Asio, then one solution is to fork the daemon process when it is still single threaded. The child process will remain single threaded and perform fork() and exec() when it is told to do so by the parent process via inter-process communication. This solution is suggested and demonstrated in this answer.

Upvotes: 3

Ulfalizer
Ulfalizer

Reputation: 4752

If you're referring to the socket descriptors themselves being inherited by exec'd child processes, and this being undesirable, then you could pass SOCK_CLOEXEC when creating the sockets with socket(2) to make sure that they are closed when you execute other programs. (This won't close the connection by the way, since your program still has a reference to the socket.)

If you are using some higher-level library, then check if there is some way to get it to pass this flag, or do fcntl(sock_fd, F_SETFD, fcntl(sock_fd, F_GETFD) | FD_CLOEXEC) to set the close-on-exec flag on the descriptor after it is created if you can get to it. (The fcntl(2) approach can be racy in a multi-threaded environment though, since some thread could exec(3) a program between the point where the socket is created and when FD_CLOEXEC is set on it.)

If the above won't work, then you could manually fork(2) and then close(2) the socket descriptors before executing a service. The advantage of SOCK_CLOEXEC is that the sockets will only be closed if the exec*() is actually successful, which sometimes makes it easier to recover from errors. In addition, SOCK_CLOEXEC can avoid some races and makes it harder to forget to close descriptors.

Upvotes: 7

Maxim Egorushkin
Maxim Egorushkin

Reputation: 136385

I have a Linux service (daemon) that has multiple-threads and uses boost io_service listening on a TCP socket. When I receive a certain message on that socket I want to start another service with e.g. /etc/init.d/corosync start

For modern Linux'es with systemd you may like to look into socket activation. That is, your main service daemon will just send a datagram to a unix socket (that datagram may contain file descriptors you would like to explicitly pass to the other service). systemd would start that other service for you if has not started yet.

The benefit of using systemd is that your service does not need to run as root in order to be able to start another service and the service processes are completely isolated from each other (no inheritance of anything as when forking).

Upvotes: 0

Related Questions