CODEWITHSUNDEEP
socketsgoudpbuffer
FanManPro
FanManPro

Reputation: 1155

How does UDP SetWriteBuffer and SetReadBuffer the OS's buffers?

Description

I'm busy writing a high frequency UDP server with Go. I'd estimate at least 1000 packets/second both ways.

However as the size of data I'm sending over the UDP socket grew, I eventually ran into the follow error: read udp 127.0.0.1:1541->127.0.0.1:9737: wsarecv: A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself.

I eventually just grew the size of the buffers I was reading from and writing into as follows:

buffer := make([]byte, 64 * 1024 * 1024) // used to just be 1024
l, err := s.socketSim.Read(buffer)

This worked fine and I stopped getting the error... However then I can across two functions inside the net package:

s.socketSim.SetWriteBuffer(64 * 1024 * 1024)
s.socketSim.SetReadBuffer(64 * 1024 * 1024)

I learned that these two act on the operating system's transmit buffer

Question

Do I even care to set the operating system buffer size and why? How does the size on the application buffer impact the size of the operating system buffer? Should they always be the same and how big should/can they become?

Upvotes: 0

Views: 1504

Answers (1)

vincent
vincent

Reputation: 1390

First, not only do you have an MTU size for each interface on your device and whatever destination you're send/recving from, but there is also an MTU size for each device in between. For this reason, as others have mentioned, you might want to use what is generally accepted for MTU since you might not control every device in the data route. In the case of UDP, MTU really just means how big a datagram can be before fragmenting.

Second, you almost certainly want your SND/RCV buffers to be larger than the MTU. These are kernel buffers which hold on to data when you're not ready to receive them. A larger UDP RCV buffer means that the kernel will buffer more packets for you instead before dropping them into the abyss. Maybe you have some non-trivial work to do for each packet. Depending on the bitrate, you might want a larger or smaller kernel buffer.

Finally, you're using UDP. There is no guarantee that you'll receive packets in order or at all. Any router in between you and a peer could decide to drop the packet for any reason. Since you're using UDP, you should prepare for dropped and out-of-order packets. You also might need some sort of retransmission mechanism, which further complicates things.

Or you might consider using TCP if dropped packets are unacceptable, knowing that timing is indeterminate.

If you're on linux, you can see current buffer sizes in /proc/sys/net. Usually the kernel will double what you ask for.

Also, you can tune your buffer size by watching for packet drops in /proc/net/udp. If you see drops, you might want to make your rcv buffer bigger, especially if the data is bursty and the processing intensive. If you're data is coming in at a consistent rate and you're still dropping packets, then you aren't processing them fast enough.

Upvotes: 1

Related Questions