how to test user space tcp/ip stack?

Question

I am working on a user space tcp stack (mostly just for fun) and I am having some trouble testing it against the unix tcp stack. Currently the only form of testing is done via unit tests. However, I want to test my tcp stack against a real kernel tcp stack. I tried the following setups without much success.

• using raw sockets: I wrote a simple echo tcp server that accepts connection using the kernel tcp socket. The tcp server listens to port 8080 on localhost. My tcp client uses the user space tcp stack. However, the kernel sends a tcp rst whenever the client sends a syn to the server. It kind of work after I modified iptable to drop all tcp rst packets. However, even though the 3 way syn, syn+ack, ack handshake is established, the server cannot recv any packet that my client sends. I eventually gave up on raw sockets.

• using tun/tap: Similarly the echo server uses kernel tcp socket and listens on localhost port 8080. The client opens a tap device. The tap device has an ip of 10.0.0.1 and my client assumes an ip of 10.0.0.2. I am able to ping 10.0.0.2 from my computer. However, when my client sends a syn to the tcp server over the tap device, the server does not respond.

Note: I am using ubuntu 12.04.

Answer 1

You can use the conntrack tool to try getting more information on why it's not working with using raw sockets. If for some reason the kernel gets confused about the state of the tcp connection, it may be deciding to reset it. You could try telling the kernel not to track connections to rule this out by setting a notrack rule in the raw table. Something like

iptables -t raw -A PREROUTING -p tcp --port 8080 -j NOTRACK

Try using tcpdump on the tun/tap device and iptables counts to see where the packet gets dropped. I would also try tun devices instead so you only have to worry about layer 3.