trace hard irqs in Linux

Question

My experience with the Linux kernel is very minimal. I have just started to play around with it recently.

I have been trying to trace the earliest time of arrival of a packet for my research purpose. I can do that at the device driver level by modifying the device driver and recording the timestamps in the device driver's interrupt handler function. I am sorry that this post might be a little bit longer.

For example, I have modified this function (https://elixir.bootlin.com/linux/v4.7/source/drivers/net/ethernet/intel/i40e/i40e_main.c#L3232) to trace the timestamp of the invocation of this function.

diving further deep and following the stack trace of this invocation, we will find a stack trace as below:

• i40e_msix_clean_rings() - i40e driver's i40e_main.c in the link provided above
• __handle_irq_event_percpu() - kernel/irq/handle.c
• handle_irq_event_percpu() - kernel/irq/handle.c
• handle_irq_event() - kernel/irq/handle.c
• handle_edge_irq() - kernel/irq/chip.c
• handle_irq() - arch/x86/kernel/irq_64.c
• do_IRQ() - arch/x86/kernel/irq.c
• common_interrupt() - not pretty sure, but implementation should be similar to early_idt_handler_common() in arch/x86/kernel/head_32.s

I am trying to trace the timestamp of the arrival of my packet in the function do_IRQ() function (boldface in stack trace above). For reference, the do_IRQ function looks like:

__visible unsigned int __irq_entry do_IRQ(struct pt_regs *regs)
{
    struct pt_regs *old_regs = set_irq_regs(regs);
    struct irq_desc * desc;
    /* high bit used in ret_from_ code  */
    unsigned vector = ~regs->orig_ax;
    **int int_number;**

    entering_irq();

    /* entering_irq() tells RCU that we're not quiescent.  Check it. */
    RCU_LOCKDEP_WARN(!rcu_is_watching(), "IRQ failed to wake up RCU");

    desc = __this_cpu_read(vector_irq[vector]);
    **int_number = desc->irq_data.irq;**

    **printk(KERN_INFO "IRQ Number=%d; Vector=%d \n", int_number, vector);**

    if (!handle_irq(desc, regs)) {
        ack_APIC_irq();

        if (desc != VECTOR_RETRIGGERED) {
            pr_emerg_ratelimited("%s: %d.%d No irq handler for vector\n",
                         __func__, smp_processor_id(),
                         vector);
        } else {
            __this_cpu_write(vector_irq[vector], VECTOR_UNUSED);
        }
    }

    exiting_irq();

    set_irq_regs(old_regs);
    return 1;
}

To make my intentions a bit clearer, I have indicated my changes in this function wrapped in " ** ".

For example, on my test machine, my NIC is bound to IRQ Number 19. The int_number variable represents that number. Thus, this lets me track the IRQ for a particular IRQ number.

It might not sound relevant for a single queue NIC Adapter, but it will be applicable for a multi-queue adapter, as I can direct my packets to a fixed queue with flow director and each queue is bound to a specific IRQ number. Thus, this will help me trace my packets easily.

My approaches:

1. adding manual implementations inside this function; which I do not think is the right approach.
2. using kprobes. But does it let me filter my traces based on the contents inside of the variables or the arguments?
3. using jprobe. I guess, with this approach, we will be able to play with the arguments. I was able to handle the event though. I simply followed the examples of jprobe. (https://stuff.mit.edu/afs/sipb/contrib/linux/samples/kprobes/jprobe_example.c) and several more.
4. I came across other tools as well, while going through the approaches mentioned above. like perf, perf-tools, eBPF. But, I am not sure which would be the best approach for me.

Just to clarify my final task: I am trying to capture the timestamps of the earliest arrival of my packets like:

t1
t2
t3
t4 

I would appreciate any kind of inputs on this.

Thank you !