CODEWITHSUNDEEP
clinuxsocketsbpfxdp-bpf
binaryBigInt
binaryBigInt

Reputation: 1704

AF_XDP: `BPF_MAP_TYPE_XSKMAP` only has entries with `Operation not supported`

This is all my XDP / BPF kernel code:

struct bpf_map_def SEC("maps") xsks_map = {
    .type = BPF_MAP_TYPE_XSKMAP,
    .key_size = sizeof(int),
    .value_size = sizeof(int),
    .max_entries = 64,  /* Assume netdev has no more than 64 queues */
};

struct bpf_map_def SEC("maps") rx_queue_pckt_counter_map = {
    .type = BPF_MAP_TYPE_ARRAY,
    .key_size = sizeof(int),
    .value_size = sizeof(unsigned long),
    .max_entries = 48,
};

SEC("xdp_sock")
int xdp_sock_prog(struct xdp_md *ctx) {

    int index = ctx->rx_queue_index;

    void *data_end = (void *)(long)ctx->data_end;
    void *data = (void *)(long)ctx->data;

    void *pos = data;
    struct ethhdr *eth = (struct ethhdr*)(pos);

    if(eth + sizeof(struct ethhdr) <= data_end) {

        if(bpf_ntohs(eth->h_proto) == ETH_P_IP) {

            struct iphdr *iph = (struct iphdr*)(pos + sizeof(struct ethhdr));

            if(iph + sizeof(struct iphdr) <= data_end) {

                if(iph->protocol == IPPROTO_UDP) {

                    const __u16 iph_sz_in_bytes = iph->ihl * 4;

                    if(iph + iph_sz_in_bytes <= data_end) {
                        struct udphdr *udh = (struct udphdr*)(pos + sizeof(struct ethhdr) + iph_sz_in_bytes);

                        if(udh + sizeof(struct udphdr) <= data_end) {

                            void *rec = bpf_map_lookup_elem(&rx_queue_pckt_counter_map, &index);
                            if(rec) {
                                long *pckt_counter_val = (long*)(rec);
                                *pckt_counter_val += 1;
                            } else {
                                return XDP_PASS;
                            }

                            if (bpf_map_lookup_elem(&xsks_map, &index)) {

                                const int ret_val = bpf_redirect_map(&xsks_map, index, 0);
                                bpf_printk("RET-VAL: %d\n", ret_val);
                                return ret_val;
                            }
                        }
                    }
                }
            }
        }
    }

    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

I am trying to filter all IP-UDP packets and send them into user-space. I also count the number of packets arriving for each RX-Queue (indicated by ctx->rx_queue_index).

My program compiles fine but for whatever reason I don't receive any packets in my user-space program. I already talked about this in another post of mine: AF_XDP: No packets from multicast although steered on RX-Queue 0

I executed sudo ethtool -N eth20 flow-type udp4 action 0 beforehand to steer all packets onto RX-Queue 0.

I can see all bpf maps currently active via 

$ sudo bpftool map list       
32: lpm_trie  flags 0x1
        key 8B  value 8B  max_entries 1  memlock 4096B
33: lpm_trie  flags 0x1
        key 20B  value 8B  max_entries 1  memlock 4096B
34: lpm_trie  flags 0x1
        key 8B  value 8B  max_entries 1  memlock 4096B
35: lpm_trie  flags 0x1
        key 20B  value 8B  max_entries 1  memlock 4096B
36: lpm_trie  flags 0x1
        key 8B  value 8B  max_entries 1  memlock 4096B
37: lpm_trie  flags 0x1
        key 20B  value 8B  max_entries 1  memlock 4096B
125: array  name rx_queue_pckt_c  flags 0x0
        key 4B  value 8B  max_entries 48  memlock 4096B
126: xskmap  name xsks_map  flags 0x0
        key 4B  value 4B  max_entries 64  memlock 4096B

But I think only 125 and 126 are related to my program.

The queue-steering works because with sudo bpftool map dump id 125 I get:

key: 00 00 00 00  value: 99 1a cc 04 00 00 00 00
key: 01 00 00 00  value: 00 00 00 00 00 00 00 00
key: 02 00 00 00  value: 00 00 00 00 00 00 00 00
key: 03 00 00 00  value: 00 00 00 00 00 00 00 00
key: 04 00 00 00  value: 00 00 00 00 00 00 00 00
key: 05 00 00 00  value: 00 00 00 00 00 00 00 00
key: 06 00 00 00  value: 00 00 00 00 00 00 00 00
key: 07 00 00 00  value: 00 00 00 00 00 00 00 00
key: 08 00 00 00  value: 00 00 00 00 00 00 00 00
key: 09 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0a 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0b 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0c 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0d 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0e 00 00 00  value: 00 00 00 00 00 00 00 00
key: 0f 00 00 00  value: 00 00 00 00 00 00 00 00
key: 10 00 00 00  value: 00 00 00 00 00 00 00 00
key: 11 00 00 00  value: 00 00 00 00 00 00 00 00
key: 12 00 00 00  value: 00 00 00 00 00 00 00 00
key: 13 00 00 00  value: 00 00 00 00 00 00 00 00
key: 14 00 00 00  value: 00 00 00 00 00 00 00 00
key: 15 00 00 00  value: 00 00 00 00 00 00 00 00
key: 16 00 00 00  value: 00 00 00 00 00 00 00 00
key: 17 00 00 00  value: 00 00 00 00 00 00 00 00
key: 18 00 00 00  value: 00 00 00 00 00 00 00 00
key: 19 00 00 00  value: 00 00 00 00 00 00 00 00
key: 1a 00 00 00  value: 00 00 00 00 00 00 00 00
key: 1b 00 00 00  value: 00 00 00 00 00 00 00 00
key: 1c 00 00 00  value: 00 00 00 00 00 00 00 00
key: 1d 00 00 00  value: 00 00 00 00 00 00 00 00
key: 1e 00 00 00  value: 00 00 00 00 00 00 00 00
key: 1f 00 00 00  value: 00 00 00 00 00 00 00 00
key: 20 00 00 00  value: 00 00 00 00 00 00 00 00
key: 21 00 00 00  value: 00 00 00 00 00 00 00 00
key: 22 00 00 00  value: 00 00 00 00 00 00 00 00
key: 23 00 00 00  value: 00 00 00 00 00 00 00 00
key: 24 00 00 00  value: 00 00 00 00 00 00 00 00
key: 25 00 00 00  value: 00 00 00 00 00 00 00 00
key: 26 00 00 00  value: 00 00 00 00 00 00 00 00
key: 27 00 00 00  value: 00 00 00 00 00 00 00 00
key: 28 00 00 00  value: 00 00 00 00 00 00 00 00
key: 29 00 00 00  value: 00 00 00 00 00 00 00 00
key: 2a 00 00 00  value: 00 00 00 00 00 00 00 00
key: 2b 00 00 00  value: 00 00 00 00 00 00 00 00
key: 2c 00 00 00  value: 00 00 00 00 00 00 00 00
key: 2d 00 00 00  value: 00 00 00 00 00 00 00 00
key: 2e 00 00 00  value: 00 00 00 00 00 00 00 00
key: 2f 00 00 00  value: 00 00 00 00 00 00 00 00
Found 48 elements

As you can see, only the counter for RX-Queue 0 is greater than 0.

But, if I take a look at the BPF_MAP_TYPE_XSKMAP (which is used to transfer packets into user-space), I get:

$ sudo bpftool map dump id 126
key:
00 00 00 00
value:
Operation not supported
key:
01 00 00 00
value:
Operation not supported
key:
02 00 00 00
value:
Operation not supported
key:
03 00 00 00
value:
Operation not supported
key:
04 00 00 00
value:
Operation not supported
...
key:
3e 00 00 00
value:
Operation not supported
key:
3f 00 00 00
value:
Operation not supported
Found 0 elements

Is the message Operation not supported an indicator why I don't receive any packets in my user-space program? Or is it just not possible to receive the value during runtime? I also find it strange to see Found 0 elements.

Any ideas what is wrong here?

Upvotes: 2

Views: 732

Answers (1)

Qeole
Qeole

Reputation: 9114

This is simply because maps of type BPF_MAP_TYPE_XSKMAP do not support lookup from user space (you'd get addresses from kernel space, which does not make sense from user space point of view and might be a security issue).

So because attempting the lookup returns -EOPNOTSUPP, bpftool is unable to show the values. It could error out and print nothing, but instead we made it print the keys it finds, and print the error messages we get for the values.

As for the Found 0 elements, the count is for the elements that bpftool could retrieve without any error, so it's only logical that it remains at zero in this case.

So nothing seems wrong in your case, I do not believe this output is related to your issue with the missing packets.

Upvotes: 2

Related Questions