Choppy audio with NAudio using when trying to simulate a gameloop

Question

I am trying to simulate a gameloop in NAudio, I have two loops one for recording and one for playing the audio back. Playback loop works every ~16ms but it sounds weird and choppy.

Here is the code i'm using

static void PlaybackLoop(double dt)
        {
            int tickSample = 960;
            short[] toPlay = new short[tickSample];

            if (waitingToPlay.Count > 0)
            {
                long elapsed = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() - lastPlayData;
                lastPlayData = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
                for (int i = 0; i < tickSample; i++)
                {
                    toPlay[i] = waitingToPlay.Count > 0 ? waitingToPlay.Dequeue() : (short)0;
                }
                // Console.WriteLine(toPlay.Length);
                // Console.WriteLine(tickSample * 12);

                Console.WriteLine("Volume: " + AvgData(toPlay) + " Length: " + toPlay.Length + " Queue: " + waitingToPlay.Count + " deltatime: " + dt);
                byte[] raw = new byte[toPlay.Length * sizeof(short)];
                Buffer.BlockCopy(toPlay, 0, raw, 0, toPlay.Length);
                bufferedWaveProvider.AddSamples(raw, 0, raw.Length);
                _previousTickVoicePlayed = true;
            }

        }
        static void Initialize()
        {
            NAudio.Wave.WaveInEvent sourceStream = new NAudio.Wave.WaveInEvent();
            sourceStream.WaveFormat = new NAudio.Wave.WaveFormat(48000, 16, 1);
            sourceStream.DataAvailable += new EventHandler<NAudio.Wave.WaveInEventArgs>(sourceStream_DataAvailable);
            sourceStream.StartRecording();

            waitingToPlay = new Queue<short>();

            NAudio.Wave.WaveOutEvent outputStream = new NAudio.Wave.WaveOutEvent();
            bufferedWaveProvider = new BufferedWaveProvider(sourceStream.WaveFormat);
            outputStream.Init(bufferedWaveProvider);
            outputStream.Play();
        }

        private static void sourceStream_DataAvailable(object sender, WaveInEventArgs e)
        {
            short[] sdata = new short[(int)Math.Ceiling(e.BytesRecorded / 2d)];
            Buffer.BlockCopy(e.Buffer, 0, sdata, 0, e.BytesRecorded);
            int countFirst = waitingToPlay.Count;
            foreach (short s in sdata)
            {
                waitingToPlay.Enqueue(s);
            }
            int countAfter = waitingToPlay.Count;
            Console.WriteLine((DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() - LastRecordTime) + " Record ms " + (countAfter - countFirst) + " sample ");
            LastRecordTime = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
        }

        static void Main(string[] args)
        {
            Initialize();
            _previousGameTime = DateTimeOffset.Now.ToUnixTimeMilliseconds();
            while (true)
            {
                long elapsedTime = (DateTimeOffset.Now.ToUnixTimeMilliseconds() - _previousGameTime);
                _previousGameTime = DateTimeOffset.Now.ToUnixTimeMilliseconds();
                double dt = elapsedTime / 1000f;
                // Update the game
                PlaybackLoop(dt);
                // Update Game at 60fps
                Task.Delay(8).Wait();
            }
        }

I tried to change tickSample based on dt but it didn't worked also. I guess i need to do something with waveout but i'm not sure what i need to do, any help is appreciated thanks

Answer 1

There are numerous issues that could cause dropouts.

The most serious issue is that the Queue is not thread safe. I suspect the second check of waitingToPlay.Count > 0 is indicative of this. You really want to be using ConcurrentQueue.

Next, you'll find that most audio software processes data in blocks for efficiency. Writing and reading the queue a single sample at a time is not going to give you that. Instead, try changing the type of queue to a short[]. This would also aid the playback loop by providing the sample array from the queue that could be written out directly without a allocation and copy.

Thirdly, the playback loop is a busy wait. When there are no samples in the queue it is spinning and wasting a lot of CPU resources. Instead, you want the thread to be idle until there is something in the queue. For this you can wrap your ConcurrentQueue in a BlockingCollection or wait on a ManualResetEvent which is signaled high when the queue has something in it.

You'll probably want to ditch the Console.WriteLine calls as they can interfere.

Doing these things will get you a good amount of the way there. The last issue I'll point out is audio applications typically have some input to output latency. Meaning, the output samples are played back by some number of samples later than they were received. The reason for this is that Windows is not a real-time operating system. And C# has a garbage collector. The computer can go out to lunch between the time you put samples into the queue and the time you write to the audio output. If this happens then the audio output DAC is starved for samples and you get a glitch. To add latency you wait until you've received a number of samples before you start playback. A lot of audio apps give the user control over the latency so the user can find the best (lowest) latency without dropouts on their system.