How can a called external program unexpectedly interfere with the shell's variables in bash/sh/ash like this?

Question

I have had a baffling issue for years where some programs (like ffmpeg) called from a while-loop in bash, sh or busybox ash interfere with either the read built-in or the shell itself, consistently on every other iteration.

The behaviour can be demonstrated as follows (change test.mkv to any valid video file path):

#!/bin/bash
while IFS= read -rd '' f;do
    ls -- "$f"
    ffmpeg -i "$f" -c copy -f matroska - &>/dev/null
done < <(printf '%s\0' test.mkv{,,,})

On every other iteration, $f gets mangled like this (it seems to always be a couple of bytes missing from the start):

test.mkv
ls: cannot access '.mkv': No such file or directory
test.mkv
ls: cannot access 'st.mkv': No such file or directory

I don't understand at all. How is this even possible?
It does not matter how the input is fed, results are the same when omitting the subshell (erasing everything after done) and externally piping something like find -print0 to the script.

Using readarray and iterating over the array in a for-loop fixes the issue at the cost of time and memory to store all of the inputs at once:

#!/bin/bash
readarray -d '' arr < <(printf '%s\0' test.mkv{,,,})
for f in "${arr[@]}";do
    ls -- "$f"
    ffmpeg -i "$f" -c copy -f matroska - &>/dev/null
done

output:

test.mkv
test.mkv
test.mkv
test.mkv