iOS AudioQueue stuttering

Question

I am building a streaming system for audio playback, and every so often, the audio either glitches or starts stuttering for a second or two.

I am running a single output audioQueue with 3 allocated buffers of 1024 samples with a sample rate of 22050.

I hold a separate list of buffers ready to stream, and that buffer is never empty (logs always show at least one filled buffer there whenever a playback_callback is called). playback_callback just memcpy-s a ready buffer into one of the three AudioBuffers with no locks or other weirdness.

playback_callback takes at most 0.9ms to run (measured via mach_absolute_time), which is far below the 1.0/(22050/1024) = 46.4 ms.

I initiate the queue with either CFRunLoopGetMain () or NULL (which should use an "internal thread") and get the same behavior in both cases.

If buffer size is turned absurdly high (16384 instead of 1024), glitches go away. If the nuber of AudioBuffers is turned from 3 to 8, it practically goes away (happens ~20x more rarely). However, neither of those settings is workable for me as it is not ok for the system to take a second to react to a stream switch (0.1 - 0.2s would still be tolerable)

Any help and ideas on the matter would be greatly appreciated.