CODEWITHSUNDEEP
multithreadingaudioaudio-streamingdecoding
olivierg
olivierg

Reputation: 10240

Which thread to use for audio decoding?

When working with audio playback I am used to the following pattern:

This works fine, and there's no issue when working with, say, a WAV file.

Now, if the source data is encoded in a compressed format, like Vorbis or MP3, decoding takes some time.

And it seems like it's quite common to perform decoding in the disk/network thread.

But isn't this wrong design? While disk or network access blocks, some CPU time is available for decoding, but is wasted if decoding happens in the same thread.

It seems to me that if the networks becomes slow, then risks of buffer underruns are higher if decoding happens sequentially.

So, shouldn't decoding be performed in the audio thread?

In my context, I would prefer to avoid adding a dedicated decoding thread. It's for mobile platforms and SMP is pretty rare right now. But please tell if a dedicated decoding thread really makes sense to you.

Upvotes: 5

Views: 1172

Answers (2)

Lior Kogan
Lior Kogan

Reputation: 20628

If your device has a single CPU, all threads are sharing it. OS Thread swapping is usually very efficient (you won't lose any meaningfull CPU power for the swapping). Therefore, you should create more threads if it will simplify your logic.

In your case, there is a pipeline. Different thread for each stage of the pipeline is a good pattern. The alternative, as you notice, involves complex logic, synchronizations, events, interrupts, or whatever. Sometimes there is no good alternatives at all.

Hence, my suggestion - create a dedicated thread for the audio decoding.

If you'll have more than a single CPU, you'll even gain more efficiency by using one thread for each pipeline step.

Upvotes: 2

Samuel Neff
Samuel Neff

Reputation: 74909

It's more important for the audio thread to be available for playing audio smoothly than for the network thread to maintain a perfect size buffer. If you're only using two threads, then the decoding should be done on the network thread. If you were to decode on the playing thread then it's possible the time could come that you need to push more audio out to the hardware but the thread is busy decoding. It's better if you maintain a buffer of already decoded audio.

Ideally you would use three threads. One for reading the network, one for decoding, and one for playing. In our application that handles audio/video capture, recording, and streaming we have eight threads per stream (recently increased from six threads since we added new functionality recently). It's much easier for each thread to have it's own functionality and then it can appropriately measure its performance against those of it's incoming/outgoing buffers. This also benefits profiling and optimization.

Upvotes: 2

Related Questions