Why on receiving SYN/ACK, a RST packet is sent with certain sites

Question

I'm running ArchLinux, and recently run into this weird problem. After some time, connecting to Google will time out because my system keeps sending a RST packet on receiving SYN/ACK packet from the server. It is the same with other IPs and port numbers of Google. It also happens with yahoo.com.

This never happened before. I think there may be something wrong with my system, but I don't remember I have changed the system config recently.

[UPDATE]

This happened again and I got the following tcpdump output as sugguested by skjaidev:

$ sudo tcpdump -i eth0 "ip host 209.85.153.100"
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
13:44:15.737180 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [S], seq 1037252805, win 14600, options [mss 1460,sackOK,TS val 33830667 ecr 0,nop,wscale 6], length 0
13:44:15.905741 IP encrypted.google.com.https > 10.20.1.113.53894: Flags [S.], seq 371070936, ack 1037252806, win 5672, options [mss 1430,sackOK,TS val 266521270 ecr 33818028,nop,wscale 6], length 0
13:44:15.905761 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [R], seq 1037252806, win 0, length 0
13:44:16.209090 IP encrypted.google.com.https > 10.20.1.113.53894: Flags [S.], seq 371070936, ack 1037252806, win 5672, options [mss 1430,sackOK,TS val 266521573 ecr 33818028,nop,wscale 6], length 0
13:44:16.209111 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [R], seq 1037252806, win 0, length 0
13:44:18.738936 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [S], seq 1037252805, win 14600, options [mss 1460,sackOK,TS val 33831568 ecr 0,nop,wscale 6], length 0
13:44:18.896373 IP encrypted.google.com.https > 10.20.1.113.53894: Flags [S.], seq 417800121, ack 1037252806, win 5672, options [mss 1430,sackOK,TS val 266524260 ecr 33818028,nop,wscale 6], length 0
13:44:18.896391 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [R], seq 1037252806, win 0, length 0
13:44:24.752266 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [S], seq 1037252805, win 14600, options [mss 1460,sackOK,TS val 33833372 ecr 0,nop,wscale 6], length 0
13:44:25.102758 IP encrypted.google.com.https > 10.20.1.113.53894: Flags [S.], seq 514770952, ack 1037252806, win 5672, options [mss 1430,sackOK,TS val 266530467 ecr 33818028,nop,wscale 6], length 0
13:44:25.102777 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [R], seq 1037252806, win 0, length 0
13:44:36.765603 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [S], seq 1037252805, win 14600, options [mss 1460,sackOK,TS val 33836976 ecr 0,nop,wscale 6], length 0
13:45:28.236165 IP 10.20.1.113.53890 > encrypted.google.com.https: Flags [P.], seq 31651351:31651378, ack 2535469155, win 279, options [nop,nop,TS val 33852417 ecr 299514426], length 27
13:45:28.236184 IP 10.20.1.113.53890 > encrypted.google.com.https: Flags [F.], seq 27, ack 1, win 279, options [nop,nop,TS val 33852417 ecr 299514426], length 0
13:45:28.426021 IP encrypted.google.com.https > 10.20.1.113.53890: Flags [F.], seq 1, ack 27, win 194, options [nop,nop,TS val 299629642 ecr 33852417], length 0
13:45:28.426044 IP 10.20.1.113.53890 > encrypted.google.com.https: Flags [.], ack 2, win 279, options [nop,nop,TS val 33852474 ecr 299629642], length 0
13:45:28.426983 IP encrypted.google.com.https > 10.20.1.113.53890: Flags [.], ack 28, win 194, options [nop,nop,TS val 299629644 ecr 33852417], length 0

Answer 1

Your clock is most likely drifting. Setup NTPD to be more aggressive in updating your system time so that you will always be more accurate. Especially if your system goes into hibernation often. The clock will always fall off time.

Answer 2

I finally find a solution for this---disabling TCP timestamps (sysctl net.ipv4.tcp_timestamps=0) solves the problem for me. The timestamps for those packets seem invalid somehow.

Answer 3

Notice how the source ports, sequence number, etc. are same in these two packets?

13:44:16.209111 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [R], seq 1037252806, win 0, length 0
13:44:18.738936 IP 10.20.1.113.53894 > encrypted.google.com.https: Flags [S], seq 1037252805, win 14600, options [mss 1460,sackOK,TS val 33831568 ecr 0,nop,wscale 6], length 0

You can't reuse source-ports that quick. There is something going on with your machine. Looks like your TCP stack isn't seeing any packets from the server, something else is generating the RSTs. Either you have iptable rules or there could be checksum errors, a full packet capture (use -s 0 -w foo.pcap and analyze the trace in wireshark) should show you any checksum errors.