CODEWITHSUNDEEP
javascriptaudioaudio-streamingaudio-recordingmedia-source
danielbernatek
danielbernatek

Reputation: 43

JavaScript Media Source API - Crackling audio

is there anybody experienced with MediaSource API and audio streaming?

I am trying to make a web app for webinars. What I have done already and working:

1) On the streamer side I create a media stream from mic input, pass
it to the media recorder.

2) From media recorder I get blob each 5 seconds and through
WebSockets send it to the server and then to the participants.

3) On the participant side I make an array buffer from each blob
(using arrayBuffer() method on the blob) and pass it to the Media
Source appendBuffer() method.

4) I try to play.

Now, what is the problem...

Everything works great but the sound played from Media Source buffer via tag is crackling. I am not talking about gaps between chunks. It has not still the same frequency and sometimes it crackles more and sometimes less or do not at all.

I was debugging many hours and find out that just playing blobs (on the both streamer and participant side) is without crackling.

So I guess the problem lies either in converting blob to array buffer or in appending it to the Media Source. Actually, I tried to convert the array buffer back to the blob and then tried to play and there was no crackling. It really looks like the problem appears after it is loaded into Media Source buffer.

I cannot find any similar questions or articles on the internet and the documentation is pretty weak in my opinion, so I hope somebody here will be able to help me.

I am using MediaSource for webcam video stream too - video is absolutely OK, audio is crackling, but much less than it does in case only audio is streamed.

Some technical parameters:

MediaStream and MediaRecorder use 44100 Hz sample rate (I tried other sample rates too, when I was trying to figure out where the problem lies) and MediaRecorder output is audio/webm with Opus codec. MediaSource is expecting exactly the same media type and codec.

I recorded you a video showing the problem. In the first part I am acting as a streamer, then I play it as a participant and lastly I show you that the blob, before it is converted to array buffer and appended to the buffer, do not have the problem.

Video showing the problem

Upvotes: 2

Views: 612

Answers (1)

Lorik
Lorik

Reputation: 525

If anyone is actively experiencing this and wants a fix, it's probably caused by something called a buffer underrun/underflow which happens when the CPU does not write fast enough to the buffer causing missing ‘gaps’ creating that crackling/stuttering effect.

See https://github.com/SamuelScheit/puppeteer-stream/issues/185 for a good guide.

Upvotes: 0

Related Questions