SYN receives RST,ACK very frequently

Question

Hi Socket Programming experts,

I am writing a proxy server on Linux for SQL server 2005/2008 running on Windows. The proxy is coded using bsd sockets and in C, and it is working fine with a problem described below.

When I use a database client (written in JAVA, and running on a Linux box) to fire queries (with a concurrency of 100 or more) directly to the Database server, not experiencing connection resets. But through my proxy I am experiencing many connection resets.

Digging deeper I came to know that connection from 'DB client' to 'Proxy' always succeeds but when the 'Proxy' tries to connect to the DB server the connection fails, due to the SYN packet getting RST,ACK.

That was to give some background. The question is : Why does sometimes SYN receives RST,ACK?

DB client(linux) to Server(windows)  ----> Works fine
DB client(linux) to Proxy(Linux) to Server(windows) -----> problematic

I am aware that this can happen in "connection refused" case but this definitely is not that one. SYN flooding might be another scenario, but that does not explain fine behavior while firing to Server directly.

I am suspecting some socket option setting may be required, that the client does before connecting and my proxy does not. Please put some light on this. Any help (links or pointers) is most appreciated.

Additional info:

Wrote a C client that does concurrent connections, which takes concurrency as an argument. Here are my observations: -> At 5000 concurrency and above, some connects failed with 'connection refused'. -> Below 2000, it works fine.

But the actual problem is observed even at a concurrency of 100 or more. Note: The problem is time dependent sometimes it never comes at all and sometimes it is very frequent and DB client (directly to server) works fine at all times .

Answer 1

When the SYN receives the RST response, it should not be the problem of SQL-SERVER.

Because the application is able to accept the socket only after the tcp handshake finished.

Is there any device between the Proxy and the SQL-Server machines?

Try to make sure that the RST response come from sql-server machine.

You connection count is far from SYN-FLOOD, I think.

Answer 2

A listening socket keeps queue of established connections and connections in establishment process (e.g. SYN got, SYNACK replied, but not ACK from client yet). If a established queue overflows, IP stack reaction differs on OS. The most traditional approach was to ignore newcoming SYNs, waiting when userland accept()s and frees a slot in the queue. With SYN flooding attacks in mid-90s, the new method was invented named "SYN cookies" which drops need for establishment queue totally, in cost of need to support special TCP option. OTOH I have heard that Windows stacks change their behavior - under some conditions, the reaction to queue overflow is RST response. In earlier stacks (e.g. Win95) this was the main response and client side was correspondingly changed to ignore RST response to SYN:( That's why I guess that some proxy host feature triggers RST in Windows stack.

Another guess is that DB server closes listening socket at all under some condition (e.g. detected overload peak) which appears only with proxy.

Answer 3

SQL Server needs worker threads to accept incoming connections. If your server is worker starved (which can be easily diagnosed by a high number of entries in sys.dm_os_tasks in PENDING state) then attempting to open new connection will fail. So likely what I suspect it happens is that you're pushing to the server more workload that it can handle. You need to optimize the workload or get a beefier server.

Clients like Java client make effective use of connection pooling and, even under a high load, do not need to open new connection hence you do not see this problem, instead you see only delays in requests completion.