CODEWITHSUNDEEP
windowswinapisocketsoperating-system
Oded R.
Oded R.

Reputation: 483

UDP Packets, Seen by Wireshark, Dropped by (do not even reach) WSARecvFrom

I have quite a bewildering problem.
I'm using a big C++ library for handling some proprietary protocol over UDP on Windows XP/7. It listens on one port throughout the run of the program, and waits for connections from distant peers.

Most of the time, this works well. However, due to some problems I'd experienced, I've decided to add a simple debug print directly after the call to WSARecvFrom (the win32 function used in the library to recv datagrams from my socket of interest, and tell what IP and port they came from).
Strangely enough, in some cases, I've discovered packets are dropped at the OS level (i.e. I see them in Wireshark, they have the right dst-port, all checksums are correct - but they never appear in the debug prints I've implanted into the code).

Now, I'm fully of the fact (people tend to mention a bit too often) that "UDP doesn't guarantee delivery" - but this is not relevant, as the packets are received by the machine - I see them in Wireshark.
Also, I'm familiar with OS buffers and the potential to fill up, but here comes the weird part...

I've done some research trying to find out which packets are dropped exactly. What I've discovered, is that all dropped packets share two things in common (though some, but definitely not most, of the packets that aren't dropped share these as well):

  1. They are small. Many of the packets in the protocol are large, close to MTU - but all packets that are dropped are under 100 bytes (gross).
  2. They are always one of two: a SYN-equivalent (i.e. the first packet a peer sends to us in order to initiate communications) or a FIN-equivalent (i.e. a packet a peer sends when it is no longer interested in talking to us).

Can either one of these two qualities affect the OS buffers, and cause packets to be randomly (or even more interesting - selectively) dropped?
Any light shed on this strange issue would be very appreciated.

Many thanks.

EDIT (24/10/12):

I think I may have missed an important detail. It seems that the packets dropped before arrival share something else in common: They (and I'm starting to believe, only they) are sent to the server by "new" peers, i.e. peers that it hasn't tried to contact before.

For example, if a syn-equivalent packet arrives from a peer* we've never seen before, it will not be seen by WSARecvFrom. However, if we have sent a syn-equivalent packet to that peer ourselves (even if it didn't reply at the time), and now it sends us a syn-equivalent, we will see it.

(*) I'm not sure whether this is a peer we haven't seen (i.e. ip:port) or just a port we haven't seen before.

Does this help?
Is this some kind of WinSock option I've never heard of? (as I stated above, the code is not mine, so it may be using socket options I'm not aware of)

Thanks again!

Upvotes: 14

Views: 16397

Answers (4)

Shiv Kishore Mishra
Shiv Kishore Mishra

Reputation: 1

Faced same kind of problem on the redhat-linux as well. This turn out to be a routing issue.

RCA is as follows:

  1. True that UDP is able to reach destination machine(seen on Wireshark).
  2. Now route to source is not found so there is no reply can be seen on the Wireshark.
  3. On some OS you can see the request packet on the Wireshark but OS does not actually delivers the packets socket (You can see this socket in the netstat-nap).
  4. In these case please check ping always (ping <dest ip> -I<source ip>)

Upvotes: 0

zeeman_effect
zeeman_effect

Reputation: 108

I had a very similar issue, after confirming that the receive buffer wasn't causing drops, I learned that it was because I had the receive timeout set too low at 1ms. Setting the socket to non-blocking and not setting the receive timeout fixed the issue for me.

Upvotes: 2

selbie
selbie

Reputation: 104559

Turn off the Windows Firewall.

Does that fix it? If so, you can likely enable the Firewall back on and just add a rule for your program.

That's my most logical guess based on what you said here in your update:

It seems that the packets dropped before arrival share something else in common: They (and I'm starting to believe, only they) are sent to the server by "new" peers, i.e. peers that it hasn't tried to contact before.

Upvotes: 1

simonc
simonc

Reputation: 42175

The OS has a fixed size buffer for data that has arrived at your socket but hasn't yet been read by you. When this buffer is exhausted, it'll start to discard data. Debug logging may exacerbate this by delaying the rate you pull data from the socket at, increasing the chances of overflows.

If this is the problem, you could at least reduce the instances of it by requesting a larger recv buffer.

You can check the size of your socket's recv buffer using

int recvBufSize;
int err = getsockopt(socket, SOL_SOCKET, SO_RCVBUF,
                     (char*)&recvBufSize, sizeof(recvBufSize));

and you can set it to a larger size using

int recvBufSize = /* usage specific size */;
int err = setsockopt(socket, SOL_SOCKET, SO_RCVBUF,
                     (const char*)&recvBufSize, sizeof(recvBufSize));

If you still see data being received by the OS but not delivered to your socket client, you could think about different approaches to logging. e.g.

  • Log to a RAM buffer and only print it occasionally (at whatever size you profile to be most efficient)
  • Log from a low priority thread, either accepting that the memory requirements for this will be unpredictable or adding code to discard data from the log's buffer when it gets full

Upvotes: 3

Related Questions