UDP Packets are delayed and processed simultaneously by the OS

Question

I am working with a microcontroller and a PC which communicate data over 1GbE Ethernet, sending custom UDP packets. The data is sent every ~20ms both ways and work fairly smoothly. However, sometimes the PC seems to process the packets at the same time as a "burst", resulting in delays of >100ms (as can be seen from the Wireshark screenshot). This is not how it should work,and I cannot understand the root cause.

I have checked the network stats (netstat -s and ifconfig, which shows that there are 0 Rx/Tx errors, and no dropped packets. The Adaptive Rx/Tx coalescing is turned off and the CPU load is below 50%. There is also nothing in the syslog or dmesg

The NIC used on the PC is Intel I219-V. PC runs Ubuntu 20.04 OS and uses e1000e for the 1GbE driver.

Has anybody seen these kinds of problems? What can it be related to? I would like to avoid such behaviour and ensure smooth, continuous packet reception, rather than in a burst with a delay.