How to Find TCP Retransmissions while sniffing packets in C

Question

I've written a simple source file that can read pcap files using the libpcap library in C. I can parse the packets one by one and analyze them up to a point. I want to be able to deduce whether a TCP packet I parsed is a TCP retransmission or not. After searching extensively the web, I've concluded that in order to so, I need to track the traffic behaviour and this means also analyzing previously received packets.

What I actually want to achieve is, to do on a basic level, what the tcp.analysis.retransmission filter does in wireshark.

This is an MRE that reads a pcap file and analyzes the TCP packets sent over IPv4. The function find_retransmissions is where the packet is analyzed.

#include <pcap.h>
#include <stdio.h>
#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <sys/socket.h>
#include <stdlib.h>
#include <net/ethernet.h>
#include <string.h>

void process_packet(u_char *,const struct pcap_pkthdr * , const u_char *);
void find_retransmissions(const u_char * , int );

int main()
{
    pcap_t *handle;
    char errbuff[PCAP_ERRBUF_SIZE];
    handle = pcap_open_offline("smallFlows.pcap", errbuff);
    pcap_loop(handle, -1, process_packet, NULL);
}

void process_packet(u_char *args,
                    const struct pcap_pkthdr * header,
                    const u_char *buffer)
{
    int size = header->len;
    struct ethhdr *eth = (struct ethhdr *)buffer;
    if(eth->h_proto == 8) //Check if IPv4
    {
        struct iphdr *iph = (struct iphdr*)(buffer +sizeof(struct ethhdr));
        if(iph->protocol == 6) //Check if TCP
        {
             find_retransmissions(buffer,size);
        }
    }
}

void find_retransmissions(const u_char * Buffer, int Size)
{
    static struct iphdr  previous_packets[20000];
    static struct tcphdr  previous_tcp[20000];
    static int index = 0;
    static int retransmissions = 0;
    int retransmission = 0;
    
    struct sockaddr_in source,dest;
    unsigned short iphdrlen;
    
    // IP header
    struct iphdr *iph = (struct iphdr *)(Buffer  + sizeof(struct ethhdr));
    previous_packets[index] = *iph;
    
    iphdrlen =iph->ihl*4;

    memset(&source, 0, sizeof(source));
    source.sin_addr.s_addr = iph->saddr;
    memset(&dest, 0, sizeof(dest));
    dest.sin_addr.s_addr = iph->daddr;

    // TCP header
    struct tcphdr *tcph=(struct tcphdr*)(Buffer 
                                  + iphdrlen 
                                  + sizeof(struct ethhdr));
    previous_tcp[index]=*tcph;
    index++;
    
    int header_size =  sizeof(struct ethhdr) + iphdrlen + tcph->doff*4;
    unsigned int segmentlength;
    segmentlength = Size - header_size;
    
    /* First check if a same TCP packet has been received */
    for(int i=0;i<index-1;i++)
    {
        // Check if packet has been resent
        unsigned short temphdrlen;
        temphdrlen = previous_packets[i].ihl*4;
        
        // First check IP header
        if ((previous_packets[i].saddr == iph->saddr) // Same source IP address
            && (previous_packets[i].daddr == iph->daddr) // Same destination Ip address
            && (previous_packets[i].protocol == iph->protocol) //Same protocol
            && (temphdrlen == iphdrlen)) // Same header length
        {
            // Then check TCP header
            if((previous_tcp[i].source == tcph->source) // Same source port
                && (previous_tcp[i].dest == tcph->dest) // Same destination port
                && (previous_tcp[i].th_seq == tcph->th_seq) // Same sequence number
                && (previous_tcp[i].th_ack==tcph->th_ack) // Same acknowledge number
                && (previous_tcp[i].th_win == tcph->th_win) // Same window
                && (previous_tcp[i].th_flags == tcph->th_flags) // Same flags
                && (tcph->syn==1 || tcph->fin==1 ||segmentlength>0)) // Check if SYN or FIN are
            {                                                        // set or if tcp.segment 0
                // At this point the packets are almost identical
                //  Now Check previous communication to check for retransmission
                for(int z=index-1;z>=0;z--)
                {   
                    // Find packets going to the reverse direction
                    if ((previous_packets[z].daddr == iph->saddr) // Swapped IP source addresses
                        && (previous_packets[z].saddr ==iph->daddr) // Same for IP dest addreses
                        && (previous_packets[z].protocol == iph->protocol)) // Same protocol
                    {
                        if((previous_tcp[z].dest==tcph->source) // Swapped ports
                            && (previous_tcp[z].source==tcph->dest)
                            && (previous_tcp[z].th_seq-1 != tcph->th_ack) // Not Keepalive
                            && (tcph->syn==1          // Either SYN is set
                                || tcph->fin==1       // Either FIN is set
                                || (segmentlength>0)) // Either segmentlength >0 
                            && (previous_tcp[z].th_seq>tcph->th_seq) // Next sequence number is 
                                                                     // bigger than the expected 
                            && (previous_tcp[z].ack  != 1))  // Last seen ACK is set
                        {
                            retransmission = 1;
                            retransmissions++;
                            break;
                        }
                    }
                }
            }
        }
    }
    
    if (retransmission == 1)
    {
        printf("Retransmission: True\n");
        printf("\n\n******************IPv4 TCP Packet*************************\n"); 
        printf("   |-IP Version       : %d\n",(unsigned int)iph->version);
        printf("   |-Source IP        : %s\n" , inet_ntoa(source.sin_addr) );
        printf("   |-Destination IP   : %s\n" , inet_ntoa(dest.sin_addr) );
        printf("   |-Source Port      : %u\n",  ntohs(tcph->source));
        printf("   |-Destination Port : %u\n",  ntohs(tcph->dest));
        printf("   |-Protocol         : %d\n",(unsigned int)iph->protocol);
        printf("   |-IP Header Length : %d DWORDS or %d Bytes\n",
(unsigned int)iph->ihl,((unsigned int)(iph->ihl))*4);
        printf("   |-Payload Length   : %d Bytes\n",Size - header_size);
        
    }
    printf("Total Retransmissions: %d\n",retransmissions);
}

This approach is based on the wireshark wiki paragraph about Retransmission. I literally have clicked every page google has to offer on how to approach this analysis but this was the only thing I was able to find. The results I get are somewhat correct, some Retransmissions go unnoticed, I get a lot of DUP-ACK packets and some normal traffic gets through as well (checked with wireshark). I use the smallFlows.pcap file found here and I believe that the results that I should have, should be the same as the tcp.analysis.retransmission && not tcp.analysis.spurious_retransmission filter in wireshark. Which amounts to 88 retransmissions for this pcap. Running this code yields 45 and I can't understand why.

Sorry for the messy if statements, I tried my best to clean them up.

Answer 1

The concept of re-transmission is simple: data that was sent, was sent again.

In TCP, every transmitted byte has an identifier. If a TCP segment has 5 bytes in it (just a hypothetical example, in reality things are bigger of course), then the identifier of the first segment is the sequence number in the TCP header, +1 for the 2nd segment, ..., +4 for the 5th.

The receiver, when it wants to acknowledge a byte, it just sends an ACK with byte's sequence number +1. If receiver wants to acknowledge the 5 bytes as in our example, it ACKs the 5th byte, which is seq_num + 4 + 1. In your case, you do this calculation to get the next expected sequence number seq_num + 4 + 1.

Then, in order to detect if a re-transmission has happened, you simply know it if the same source has sent a TCP segment with a sequence number that's lower than the expected seq_num + 4 + 1.

Say, instead of getting seq_num + 4 + 1 in the next transmitted TCP message, you got seq_num. This means that the this segment is a re-transmission of the previous one.

But does it mean that this TCP segment, with the re-transmission, only contains re-transmissions? No. It can contain re-transmissions from previous segment, plus extra bytes for the next segment. This is why you need to count the total bytes in the segments to tell how many of the bytes are part of the re-transmissions, and how many are part of new transmission. As you see, TCP re-transmission is not binary per segment, but can overlap across segments. Because we are really re-transmitting bytes. We just store bytes in segments for reducing TCP header's overhead.

Now, what if you got seq_num + 2 + 1? This is a bit odd because it indicates that the previous segment got partially re-transmitted only. It basically indicates that it's only re-transmitting from byte 3. If the segment has only 3 bytes, it re-transmitting 3rd, 4th and 5th bytes (i.e. only the previous segment's bytes). But if it has, say, 10 bytes, it means that 6th, 7th, 8th, 9th and 10th bytes are new bytes (not re-transmitted).

In my opinion you can only say that a TCP packet is a re-transmission only when it's carrying bytes with identifiers that were sent before. But as said earlier, this might not be true, as a segment could contain some bytes sent earlier, plus more never sent, hence being a mixture between re-transmissions and new-transmissions.

Answer 2

For detecting a retransmission you have to keep track of the expected sequence number. If the sequence number is higher than expected the packet is a retransmitted one ( TCP Analysis chapter of the wireshark docs, https://www.wireshark.org/docs/wsug_html_chunked/ChAdvTCPAnalysis.html )

TCP Retransmission

Set when all of the following are true:

• This is not a keepalive packet.
• In the forward direction, the segment length is greater than zero or the SYN or FIN flag is set.
• The next expected sequence number is greater than the current sequence number

Beside TCP Retransmission this there is also TCP Spurious Retransmission and TCP Fast Retransmission

Basically a retransmission is only necessary if a package is lost. Analyzing lost segment inconsistency :

source of graphic : http://www.opentextbooks.org.hk/ditatopic/3578

For detecting this type of fault in wireshark the filter tcp.analysis.ack_lost_segment is used. Maybe try to implement this.

(https://serverfault.com/questions/626273/how-can-i-write-a-filter-to-get-tcp-sequence-number-inconsisten)

In wireshark several filters can be applied to capture all types of inconsistencies in sequence numbers i.e. tcp.analysis.retransmission, tcp.analysis.spurious_retransmission and tcp.analysis.fast_retransmission, for the general case of packet loss check for tcp.analysis.ack_lost_segment

https://superuser.com/questions/828294/how-can-i-get-the-actual-tcp-sequence-number-in-wireshark

By default Wireshark and TShark will keep track of all TCP sessions and implement its own crude version of Sliding_Windows. This requires some extra state information and memory to be kept by the dissector but allows much better detection of interesting TCP events such as retransmissions. This allows much better and more accurate measurements of packet-loss and retransmissions than is available in any other protocol analyzer. (But it is still not perfect)

This feature should not impact too much on the run-time memory requirements of Wireshark but can be disabled if required.

When this feature is enabled the sliding window monitoring inside Wireshark will detect and trigger display of interesting events for TCP such as :

• TCP Retransmission - Occurs when the sender retransmits a packet after the expiration of the acknowledgement.

• TCP Fast Retransmission - Occurs when the sender retransmits a packet before the expiration of the acknowledgement timer. Senders receive some packets which sequence number are bigger than the acknowledged packets. Senders should Fast Retransmit upon receipt of 3 duplicate ACKs.

...

source : https://gitlab.com/wireshark/wireshark/-/wikis/TCP_Analyze_Sequence_Numbers