CODEWITHSUNDEEP
linuxmultithreadingsockets
Bob
Bob

Reputation: 91

SO_REUSEPORT Socket Server

I've been working on a multi-threaded socket server for Linux, and I'm trying out different ways to multiplex the I/O to see which works best.

I've already got code that creates a shared epoll / socket - with EPOLLONESHOT enabled - and each thread pulls events from this and then re-arms the EPOLLONESHOT on the fd once processed.

("Processing", in this case, means reading until EAGAIN / EWOULDBLOCK and then sending back a simple reply. Basically, I'm using "ab" to test this, so it sends a HTTP GET request and I send an HTTP "200 OK" back.)

But I wanted to try out SO_REUSEPORT. So every thread has its own epoll / socket, bound to the same port. In effect, each thread is its own "mini-server" and we let the kernel load balance between them.

I do an accept(), get an fd for an incoming connection, so I add that to the epoll. Once processing is done on that fd, I naturally call close() to end the conversation.

But this seems to intermittently drop incoming accepts (and by "intermittently" I mean that it's behaving like a race condition - sometimes works, sometimes doesn't, in a random way).

Reading on this, there's apparently a known bug that there can be a race condition between accept() and close(), as the close() causes a rebalancing of things and the accept queue is just reset, so they get dropped.

I'm trying to work out a way around this problem.

One idea I had was to split the accepts from the epoll processing queue, so that closing an fd on the epoll can't wipe out the accepts on that queue.

But this doesn't logically work, as I can't have a thread both block on accept() and block on epoll_wait() at the same time. To properly multiplex, we have to block on all the events.

The way I've got it is that there's as many "mini-servers" as there are cores and each one is pinned to a core. So they're truly all running side-by-side, without context switches.

This means that though I could spawn a new thread to handle a new incoming fd - and leave the main thread to just accept() in a loop - then that somewhat defeats the purpose of pinning CPUs and the whole idea of multiplexing is to get away from the "one thread per connection" thing.

I've looked up source code for a SO_REUSEPORT server, to see how others might handle this, but all I could find was a simple demo that wasn't multithreaded / multicore.

Does anyone know how I might solve this problem make a multithreaded SO_REUSEPORT socket server actually work?

Upvotes: 0

Views: 728

Answers (1)

Bob
Bob

Reputation: 91

As I'm testing my own server code, my focus was on the server. Naturally enough.

But noting an example that I found on the Internet that sets the SO_RCVTIMEO socket option - the receive timeout - before adding the socket fd from the accept to the epoll, I also tried this and now it all runs without issue (a million requests with 1000 concurrency, with each core steady at around 30-40% usage).

I'd love to know exactly why this has fixed things, but I presume it's simply that my server wasn't being tolerant enough of delays, issues and such on the client side of the communication, so it was getting stuck on any hiccup in the communication - and then it would wait forever, as there was no timeouts.

It makes sense to add timeouts to all the operations - just in case - as network communication is never quite 100% perfect. And, as always in coding, one should treat all input as unreliable and potentially malicious anyway.

Upvotes: 1

Related Questions