CODEWITHSUNDEEP
windowssocketsnetwork-programmingtcp
Mehdi LAMRANI
Mehdi LAMRANI

Reputation: 11597

High Frequency Data Listener TCP Overload on WIndows 7

I have coded a c# based listener to retrieve data from a stock market server. The problem is, after monitoring through WireShark, TCP window gets full, meaning that the receiving host (me) has trouble processing incoming data fast enough.

I Know my data provider has disabled TCP acknowledgement from his side, so basically he just pushes TCP Packets and looks if they are "still there" : If he detects that they were not processed after a certain amount of time, connection is simply shut down.

I really don't know what to do, Ive disabled Windows 7's Auto Scale Tuning and also Heuristics, but that did not have any effect. I also noticed that there is no way to set TPC Windows Size (RWIN) on Windows 7.

I Know this is not a programming related question per se but it is somehow, because this is critical from the code point of view.

EDIT
A responder stated : "your application is not pulling data out of it fast enough" This is fairly interesting, but I have no real idea on how to optimize this :

    private class IoContext
    {
        // The socket used for the operation:
        public Socket _ipcSocket;
        // The buffer used for the operation:
        public VfxMsgBlock _ipcBuffer;

        public IoContext(Socket socket, VfxMsgBlock buffer)
        {
            _ipcSocket = socket;
            _ipcBuffer = buffer;
        }
    }

    private void InitiateRecv(IoContext rxContext)
    {
        rxContext._ipcSocket.BeginReceive(rxContext._ipcBuffer.Buffer, rxContext._ipcBuffer.WrIndex,
            rxContext._ipcBuffer.Remaining(), 0, CompleteRecv, rxContext);
    }

    private void CompleteRecv(IAsyncResult ar)
    {
        IoContext rxContext = ar.AsyncState as IoContext;
        if (rxContext != null)
        {
                int rxBytes = rxContext._ipcSocket.EndReceive(ar);
                if (rxBytes > 0)
                {
                    //Adjust the write index in the message block:
                    rxContext._ipcBuffer.WrIndex = rxContext._ipcBuffer.WrIndex + rxBytes;

                    //(...) Do Stuf here with data                        

                    rxContext._ipcBuffer.Crunch();

                    //Initiate another asynchronous read:
                    InitiateRecv(rxContext);
                }                
          }
     }

EDIT 2
In response to Len Holgate :

I changed/checked the Recv Buffer size directly via c# property (got some troubles using GetSocketOption/SetSocketOption)

I can notice a slight performance change when I vary the Socket Buffer size, ie connection drops after 10mn with a big bufer (1000000) rather than 3mn (using 10240 or less (even 0, I don't know why... weird))

Here is the first WireShark log I am getting : 

[SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1380 SACK_PERM=1 WS=128

It doesn't change whatever the Socket's receive buffer size

About Windows 7 & TCP Windows size, I meant than the re-write of TCP Management in Windows resulted in it being scaled automatically, and no registry parameter is available to set it "by hand".
However, you are right : On a per-socket basis, it is adjustable using the receive buffer size.
Source here among others

Upvotes: 1

Views: 3460

Answers (3)

Len Holgate
Len Holgate

Reputation: 21644

Have you set the socket's recv buffer to as large as you can?

See here for how to affect the TCP window size: http://msdn.microsoft.com/en-us/library/ms819736.aspx

Edited to show what I get with two win 7 machines over a 1gb link, no changes to default settings, just setting SO_RECVBUF using setsockopt() on the listen socket, window size verified from the wireshark log of the connection... The actual window size is calculated from the advertised size and the multiplier that is implied by the scaling factor.

 Set Recv  |Advertised |Scaling|Multiply| Actual TCP |Difference|
   Buf to  |  Window   |       |  By    |Window size |          |
   --------+-----------+-------+--------+----------- +----------+
   default |   8192    |   8   |    256 |    2097152 |          | 
      1024 |   1024    |   0   |      1 |       1024 |         0|
     10240 |  10240    |   0   |      1 |      10240 |         0|
  66666666 |  65535    |  10   |   1024 |   67107840 |   -441174|
 666666666 |  65535    |  14   |  16384 | 1073725440 |-407058774|
1000000000 |  65535    |  14   |  16384 | 1073725440 | -73725440|
1073725440 |  65535    |  14   |  16384 | 1073725440 |         0|
1073741823 |  65535    |  14   |  16384 | 1073725440 |     16383|
1073741824 |   8192    |   8   |    256 |    2097152 |1071644672|
2147483647 |   8192    |   8   |    256 |    2097152 |2145386495|

So, clearly, it IS possible to affect the TCP Window size on a per socket basis on Windows 7. Could you provide a link to the document that makes you believe that "As of windows 7 I just said that there is no TCP Window Size available for manual setting anymore (auto-tune netsh option only)" please.

Note that the recv buf setting must be applied to the listening socket, or the outbound socket before you issue the connect as the window size scaling is transmitted in the opening handshake.

In all cases the call to setsockopt() succeeded and an immediate call to getsockopt() reported the recv buffer to be the size that was requested in the previous call to setsockopt().

Note that although the theoretical maximum is the max value of an unsigned int, 2147483647, the actual maximum useful value appears to be 1073741823 (0x3FFFFFFF), this may vary with operating system version but bear in mind that an immediate call to getsockopt() returned the same value that had been set so it was impossible, except by looking at the wireshark log, to determine that values above 1073741823 were ignored entirely (at least from a window size perspective).

Of course this requires that the connecting peer is specifying a window scale option, even 0, to show that it accepts window scaling.

Please show us the wireshark connection log for the initial handshake packets. I can provide compiled code that you can you to test this is you want.

Upvotes: 2

usr
usr

Reputation: 171206

As an addition the the already excellent answers I would suggest the following test: Pull data out as fast as you can and discard it immediately. This allows you to see if your processing speed really is the problem, or if it is somewhere else. You can't go faster that reading and throwing the data away.

Upvotes: 1

Stewart
Stewart

Reputation: 4026

If you disable ack processing in TCP, you're not talking TCP any more. You're using some wierd frankenprotocol which looks a bit like TCP.

Typically though you should use Windows TCP auto tuning - it is pretty good, although it does depend on the peer talking TCP (which in your case it isn't).

If you need to prevent the window from becoming full your best bet is to process data faster - chances are that the window is filling because your application is not pulling data out of it fast enough. Streamlining that should be your first port of call. Using async sockets is a good start, keeping multiple reads pending in a row so that the OS can hand the data off to your app as soon as it is ready would be another step in the right direction.

Finally, if you want to use UDP, you should probably try to use UDP instead of wierd TCP.

Upvotes: 2

Related Questions