Inquiry regarding redirecting all packets to XDP

Question

#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/icmp.h>
#include <linux/udp.h>
#include <linux/in.h>
#include <linux/if_arp.h>

#define IFINDEX_2 2  // ifindex for enp129s0f0
#define IFINDEX_3 3  // ifindex for enp129s0f1

SEC("xdp_redirect")
int xdp_redirect_prog(struct xdp_md *ctx)
{
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;
    struct ethhdr *eth = data;

    // Check packet boundaries
    if ((void *)(eth + 1) > data_end)
        return XDP_DROP;

    // Check if the packet is ARP
    if (eth->h_proto == bpf_htons(ETH_P_ARP)) {
        if (ctx->ingress_ifindex == IFINDEX_2) {
            return bpf_redirect(IFINDEX_3, 0);
        } else if (ctx->ingress_ifindex == IFINDEX_3) {
            return bpf_redirect(IFINDEX_2, 0)
        }
    }

    if (eth->h_proto == bpf_htons(ETH_P_IP)) {
        // Determine incoming interface and redirect accordingly
        if (ctx->ingress_ifindex == IFINDEX_2) {
            return bpf_redirect(IFINDEX_3, 0); 
        } else if (ctx->ingress_ifindex == IFINDEX_3) {
            return bpf_redirect(IFINDEX_2, 0);
        }
    }

    return XDP_PASS; 

The server configuration:

server1(192.168.1.1)<->server2<-> server3(192.168.1.2)
enp129s0f3(192.168.1.1) <-> enp129s0f0np0 / enp129s0f1np1 <-> enp129s0f3(192.168.1.2)

As shown above, server1 and server2 are connected, and server2 and server3 are connected.

At this time, I want to send packets such as ARP, TCP, UDP, etc. from server1 to server3 through server2, and packets such as ARP, TCP, UDP, etc. from server3 to server1 through server2. At this time, I want to redirect packets in both directions through XDP from server2, but it doesn't work.

For example, when server1 sends an ARP REQUEST, it passes through server2 and is sent to server3, but there is a phenomenon in which the ARP REPLY from server3 is not sent to server1.

Answer 1

I suspect the issue you are having is due to you not updating the MAC addresses.

Typically, a NIC will filter out/ignore packets that are not addresses to its configured MAC address or the broadcast address (FF:FF:FF:FF:FF:FF).

Since ARP is all about translating IPs to MACs, the ARP request is sent to the broadcast address, so both server2 and server3 NIC will accept it. However, the reply will be sent to the MAC address of the sender in this case server1. So the NIC on server2 will discard it.

There are two ways to handle this. First, you add XDP logic to do proper MAC conversion. So whenever you forward a packet from server1 to server3 you change the source MAC to the MAC on enp129s0f1np1 and the destination to the MAC on server3/enp129s0f3. And in the other direction set the source to enp129s0f1np0 and destination to server1/enp129s0f3. In the case of ARP you need to not only update the ethernet header but also the fields in the ARP message since in my experience the "sender" field will be used to set the destination MAC on the reply.

The second, way to deal with this is to enable promiscuous mode on both interfaces of server2. To do so you execute ip link set <interface-name> promisc on. This will tell the NIC to always pass all packets regardless of the MAC address. You have to remember to do this every time you reboot.