CODEWITHSUNDEEP
linuxnetwork-programmingudp
lessju
lessju

Reputation: 168

Some issues with PACKET_MMAP

I'm using packet_mmap to read a fast stream of UDP packets. When using either of the following code segments to wait for incoming frames it works fine:

// ring[i].iov_base points to the start address of the ith frame
struct tpacket_hdr *header = (struct tpacket_hdr *) ring[i].iov_base;

// Using poll on socket to wait for data
while(!(header -> tp_status & TP_STATUS_USER))
    {
        struct pollfd pfd;
        pfd.fd      = _socket;
        pfd.events  = POLLIN | POLLERR;
        pfd.revents = 0;
        poll(&pfd, 1, -1);
    }

// Using nanosleep to wait for incoming data
while(!(header -> tp_status & TP_STATUS_USER))
    {
        struct timespec t, r;
        t.tv_nsec = 1;
        t.tv_sec = 0; 
        nanosleep(&t, &r)
    }

However when I try to busy wait (while(!(header -> tp_status & TP_STATUS_USER)) ; the statement remains True indefinitely after a few packets are read. Why is this so? Does the kernel only transfer frames to the ring buffer when a system call is issued? The input socket is initialised as so: socket(PF_PACKET, SOCK_DGRAM, htons(ETH_P_IP)).

Also, when using this code (using poll or nanosleep) it seems to be dropping packets, whilst a simple receive code using a UDP socket doesn't, making the packet_mmap implementation slower. Sometimes the dropped packets are detected by the sockets, however when using the PACKET_STATISTICS option to getsockopt:

    if (header -> tp_status & TP_STATUS_LOSING)
    {
        struct tpacket_stats stats;
        socklen_t size_sock = sizeof(tpacket_stats);
        if (getsockopt(_socket, SOL_PACKET, PACKET_STATISTICS, &stats, &size_sock) > -1)
            printf("Dropped packets: [%d, %d]\n", stats.tp_drops, stats.tp_packets);
    }

it states that no packets were dropped (output example: "Dropped packets: [0, 5]"). Does PACKET_STATISTICS behave differently on PACKET_RX_RING sockets?

The full code listing for this code is available here

Upvotes: 3

Views: 2420

Answers (1)

lessju
lessju

Reputation: 168

Just in case someone encounters this issue, busy waiting was "hanging" because tpacket_hdr was being optimized into registers and its value was not checked in memory. Marking this as volatile ensures that any changes made by the kernel to this structure are seen by the application.

The packet drops detected with PACKET_STATISTICS were for the first few packets which were being received, and the rest were due to the application not keeping up with the incoming packets because of the extra overhead caused by the system calls.

Upvotes: 4

Related Questions