XDP on veth cannot handle non-directly connected traffic

Question

I use XDP to add NAT functionality on a veth, and my topology is as follows:

When I ping 192.168.50.3 on veth0, veth-xdp performs DNAT by changing the destination IP to 172.10.1.2, allowing the packet to successfully reach ns1. However, when the ICMP reply packet returns from ns1 to veth0, the SNAT in veth-xdp does not take effect, resulting in the following outcome:

It is clear that SNAT is not taking effect. and my xdp program is like this:

SEC("xdp")
int xdp_redirect_func(struct xdp_md *ctx)
{
    void *data_end = (void *)(long)ctx->data_end;
    void *data = (void *)(long)ctx->data;
    int action = XDP_PASS;

    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
    {
        action = XDP_DROP;
        goto out;
    }

    if (eth->h_proto != htons(ETH_P_IP)) goto out;

    struct iphdr *ip = data + sizeof(struct ethhdr);
    if ((void *)(ip + 1) > data_end)
    {
        action = XDP_DROP;
        goto out;
    }

    __u8 protocol = ip->protocol;
    if (protocol != IPPROTO_TCP && protocol != IPPROTO_ICMP &&
        protocol != IPPROTO_UDP)
    {
        goto out;
    }

    __u32 old_src_ip = ip->saddr;
    __u32 old_dst_ip = ip->daddr;
    __u16 original_checksum = ip->check;

    __u32 *new_dst_ip = bpf_map_lookup_elem(&dnat_map, &old_dst_ip);
    if (new_dst_ip)
    {
        ip->daddr = *new_dst_ip;
        ip->check = update_checksum(original_checksum, (__u16)(old_dst_ip >> 16),
                                    (__u16)(*new_dst_ip >> 16));
        ip->check = update_checksum(ip->check, (__u16)(old_dst_ip & 0xFFFF),
                                    (__u16)(*new_dst_ip & 0xFFFF));
        goto out;
    }

    __u32 *new_src_ip = bpf_map_lookup_elem(&snat_map, &old_src_ip);
    if (new_src_ip)
    {
        ip->saddr = *new_src_ip;
        ip->check = update_checksum(original_checksum, (__u16)(old_src_ip >> 16),
                                    (__u16)(*new_src_ip >> 16));
        ip->check = update_checksum(ip->check, (__u16)(old_src_ip & 0xFFFF),
                                    (__u16)(*new_src_ip & 0xFFFF));
    }

out:
    return xdp_stats_record_action(ctx, action);
}

I would like to know if the XDP program mounted on the veth can only handle directly connected traffic, and how to make veth-xdp correctly process traffic forwarded by other veth interfaces？ Thanks！

Answer 1

XDP programs only run on ingress, so your return traffic won't be seen by your XDP program at veth-xdp. I would recommend to use tc instead of XDP and attach to both the ingress and egress hooks of veth-xdp. You won't get a performance improvement by using XDP anyway, not on veth devices.