CODEWITHSUNDEEP
csockets
user2085689
user2085689

Reputation: 336

domain socket fragmentation

I am using domain sockets (AF_UNIX) to communicate between two threads for inter process communication. This is chosen to work well with libev: I use it on the recv end of the domain socket. This works very well except that the data I am sending is constant 4864 bytes. I cannot afford to get this data fragmented. I always thought domain sockets won't fragment data, but as it turns out it does. When the communication is at its peak between the threads, I observe the following

Thread 1:
SEND = 4864 actual size = 4864

Thread 2:
READ = 3328 actual size = 4864

Thread 1:
SEND = 4864 actual size = 4864

Thread 2:
READ = 1536 actual size = 4864

As you can see, thread 2 received the data in fragments (3328 + 1536). This is really bad for my application. Is there anyway we can make it not fragment it? I understand that IP_DONTFRAG can be set to only AF_INET family? Can someone suggest an alternative?

Update: sendto code

ssize_t
socket_domain_writer_dgram_send(int *domain_sd, domain_packet_t *pkt) {

    struct sockaddr_un remote;
    unsigned long len = 0;
    ssize_t ret = 0;

    memset(&remote, '\0', sizeof(struct sockaddr_un));
    remote.sun_family = AF_UNIX;
    strncpy(remote.sun_path, DOMAIN_SOCK_PATH, strlen(DOMAIN_SOCK_PATH));
    len  = strlen(remote.sun_path) + sizeof(remote.sun_family) + 1;

    ret = sendto(*domain_sd, pkt, sizeof(*pkt), 0, (struct sockaddr *)&remote, sizeof(struct sockaddr_un));
    if (ret == -1) {
        bps_log(BPS_LOGGER_RD, ASL_LEVEL_ERR, "Domain writer could not connect send packets", errno);
    }
    return ret;
}

Upvotes: 1

Views: 1773

Answers (2)

hdante
hdante

Reputation: 8020

SOCK_STREAM by definition doesn't preserve message boundaries. Try again with SOCK_DGRAM or SOCK_SEQPACKET:

http://man7.org/linux/man-pages/man7/unix.7.html

On the other hand, consider that you may be passing messages larger than your architecture page size. For example, for amd64, a memory page is 4K. If that's a problem for any reason it might make sense to split the packets in 2.

Note however, that's not a real issue for the packets to arrive fragmented. It's common to have a packet assembler in the receiving end of the socket. What's wrong with implementing it ?

Upvotes: 2

user149341
user149341

Reputation:

4864 + 3328 = 8192. My guess is that you're transmitting two 4864-byte packets back to back in some cases, and it's filling an 8 KB kernel buffer somewhere. IP_DONTFRAG isn't applicable because IP is not involved here — the "fragmentation" you're seeing is happening via a completely different mechanism.

If all the data you're transmitting consists of packets, you would do well to use a datagram socket (SOCK_DGRAM) instead of a stream. This should make the send() block when the kernel buffer doesn't have sufficient space to store an entire packet, rather than allowing a partial write through, and will make each recv() return exactly one packet, so you don't need to deal with framing.

Upvotes: 2

Related Questions