how the difference between libevent version 1.4 and 2.0 influence libpcap events?

Question

I'm using libpcap and libevent in a program.

the related source codes are:

const u_int16_t RELAY_PORT = 8000;

pcap_t *create_pcap(const void *dev, pcap_style_t style)
{
    pcap_t *handle;                 /* Session handle */
    struct bpf_program fp;          /* The compiled filter */
    bpf_u_int32 mask;               /* The netmask */
    bpf_u_int32 net;                /* The IP subnet*/
    const struct pcap_pkthdr* pcap_header;   /* A pointer to pcap_pkthdr structure */
    const u_char *pcap_packet;           /* The captured packet */ 

    char interface[20];
    strcpy(interface, dev);

    /* Find the properties for the network interface */
    if (pcap_lookupnet(interface, &net, &mask, errbuf) == -1) {
      fprintf(stderr, "Pcap counldn't get netmask for device %s: %s\n", interface, errbuf);
      net = 0;
      mask = 0;
    }


    handle = pcap_open_live(interface, BUFSIZ, 0, 0, errbuf);
    if (handle == NULL) {
    fprintf(stderr, "Pcap open live capture failure: %s\n", errbuf);
            exit(1); 
    }

    sprintf(filter_exp, "tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack) && src port %d || dst port %d", RELAY_PORT, RELAY_PORT);

    /* Compile and apply the filter */
    if (pcap_compile(handle, &fp, filter_exp, 0, mask) == -1) {
    fprintf(stderr, "Pcap parse filter failure: %s\n", pcap_geterr(handle));
        exit(1);
    }

    if (pcap_setfilter(handle, &fp) == -1) {
        fprintf(stderr, "Pcap couldn't install filter: %s\n", pcap_geterr(handle));
        exit(1);
    }

    if(style == NONBLOCKING){
       if(pcap_setnonblock(handle, 1, errbuf) == -1){
            fprintf(stderr, "Pcap set non-blocking fails: %s\n", errbuf);
                exit(1);
      }
    }

   return handle;
}

//////////////////////////////////////////////////

void on_capture(int pcapfd, short op, void *arg)
{
   int res;
   printf("on capture \n");
   pcap_t *handle;
   handle = (pcap_t *)arg;
   fqueue_t* pkt_queue;

   /* put all packets in the buffer into the packet FIFO queue
   * and then process these packets 
   * */
    pkt_queue = init_fqueue();
    res = pcap_dispatch(handle, -1, collect_pkt, (u_char *)pkt_queue);
    printf("pcap_dispatch() returns %d\n", res);
    if(!res) return;
    process_packet(pkt_queue);
}

//////////////////

 int pcapfd;
 pcap_t *pcap_handle;
 struct event  pcap_ev;

 pcap_handle = create_pcap("eth0", NONBLOCKING);  
 pcapfd = pcap_get_selectable_fd(pcap_handle);
 if(pcapfd<0){
    perror("pcap_get_selectable_fd() failed!\n");
    exit(1);
 }

 if (setnonblock(pcapfd) == -1) return -1;

 base = event_init();

 event_set(&pcap_ev, pcapfd, EV_READ|EV_PERSIST, on_capture,  pcap_handle);
 event_base_set(base, &pcap_ev);
 if(event_add(&pcap_ev, NULL) == -1){
    perror("event_add() failed for pcap_ev!\n");
    exit(-1);
 }

  event_base_dispatch(base);
--------------------------------------------- 

I also register two TCP events on event_base( on_accept and on_recv.)

then I run the program on host A and host B send packets to A, meanwhile I use a tcpdump to capture packets on A (tcpdump -i eth0 port 8000 )

For comparison, I have two laptops which acts as A, I tried the program (compile and then run) on these two laptops, one with Fedora (fedora release 18) and one with Ubuntu (Ubuntu 14.04.2 LTS)

ubuntu: Linux 3.13.0-61-generic
fedora: Linux 3.11.10-100-fc18.x86_64

on ubuntu events are invoked in the following order

on capture 
pcap_dispatch() returns 0
on capture 
pcap_dispatch() returns 0
on accept 
on recv 

it is strange that the pcap_dispatch returns 0 twice. My expectation is that the when on_capture event is triggered, pcap_dispatch will catch TCP SYN packets before on_accept event is triggered (TCP packets are captured on NIC before handed over to TCP stack). But I don't know why the on_capture events are invoked twice and pcap_dispatch() returns 0.

on Fedora, the program works as expected, the pcap_dispatch() can capture packets the first time it is invoked before on_accept event.

I use ldd to check the libraries of this program on each laptop.

Fedora:

 $ldd relay 
linux-vdso.so.1 =>  (0x00007fff1d1ad000)
libevent-1.4.so.2 => /lib/libevent-1.4.so.2 (0x00007faca467d000)
libpcap.so.1 => /lib64/libpcap.so.1 (0x00000035b4a00000)
libc.so.6 => /lib64/libc.so.6 (0x00000035b0a00000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00000035cea00000)
librt.so.1 => /lib64/librt.so.1 (0x00000035b1a00000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00000035b2e00000)
/lib64/ld-linux-x86-64.so.2 (0x00000035b0200000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00000035b1600000)

ubuntu:

  $ ldd relay 
linux-vdso.so.1 =>  (0x00007ffd08bc5000)
libevent-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5 (0x00007eff35f81000)
libpcap.so.0.8 => /usr/lib/x86_64-linux-gnu/libpcap.so.0.8 (0x00007eff35d43000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007eff3597e000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007eff35760000)
/lib64/ld-linux-x86-64.so.2 (0x00007eff361c5000)

indeed, both libpcap and libevent versions are different. what are potential problems for my program when it runs on ubuntu? how can fix the unexpected problems on ubuntu? thank you!

Answer 1

how the difference between libevent version 1.4 and 2.0 influence libpcap events?

It doesn't.

indeed, both libpcap and libevent versions are different

Yes; as you indicated in your email to me, the libpcap on Fedora is libpcap 1.3.0 and the libpcap on Ubuntu is libpcap 1.5.3.

Libpcap 1.3.0 doesn't support TPACKET_V3, and libpcap 1.5.3 does. The kernel on both your Fedora machine (3.11.10-100-fc18.x86_64, according to your email) and your Ubuntu machine (3.13.0-61-generic, according to your email) both support TPACKET_V3.

how can fix the unexpected problems on ubuntu?

Don't use a timeout of 0 in the pcap_open_live() call. Due to the way TPACKET_V3 works, some bugs in how it works in older kernels (both of your kernels are "older" in that sense), and the way libpcap attempts to make non-blocking mode work, make a timeout of 0 work, and work around those bugs, a timeout of 0 may not work well. Try a timeout of, for example, 100 (for 1/10 of a second) or 10 (for 1/100 of a second).

Note that if a timeout of 0 works the way it's intended, it could be that an event for libpcap might not be delivered for an arbitrarily-long period of time, with the time period being longer the less traffic is captured, so it's rarely, if ever, a good idea to use a timeout of 0.