CODEWITHSUNDEEP
htmlaudiohtml5-audioweb-audio-api
boortmans
boortmans

Reputation: 1170

Play live audio stream - html5

I have a desktop application which streams raw PCM data to my browser over a websocket connection. The stream looks like this ...\\x00\\x00\\x02\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\xff\\xff\\xff\\xff\\....

The question is simple: can I play such a stream in HTML with the Web Audio API / WebRTC / ...?

Any suggestions are very welcome!

code edit

This code plays noise, randomly generated:

function myPCMSource() { 
    return Math.random() * 2 - 3;
}

var audioContext;

try {
    window.AudioContext = window.AudioContext || window.webkitAudioContext;
    audioContext = new AudioContext();
} catch(e) {
    alert('Web Audio API is not supported in this browser');
}

var bufferSize = 4096;
var myPCMProcessingNode = audioContext.createScriptProcessor(bufferSize, 1, 1);
myPCMProcessingNode.onaudioprocess = function(e) {
    var output = e.outputBuffer.getChannelData(0);
    for (var i = 0; i < bufferSize; i++) {
     output[i] = myPCMSource(); 
 }
}

So changing the myPCMSource() to the websocket stream input, should make it work somehow. But it doesn't. I don't get any errors, but the API is not playing any sound nor noise.

Upvotes: 3

Views: 3780

Answers (2)

user5311618
user5311618

Reputation:

Just for the record, the ScriptProcessorNode is deprecated. See the MDN article for details. The feature was replaced by AudioWorklets and the AudioWorkletNode interface.

In short, a ScriptProcessorNode runs outside of the browser's internal audio thread, which creates at least on frame (128 samples) of latency. Worse, the ScriptProcessorNode often fails to respond quickly enough, if the thread is busy, so will just randomly drop the ball every so often.

Worklets are basically task-specific workers that run in one of the browsers internal threads (paint, layout, audio etc). Audio worklets run in the audio thread, and implement the guts of custom audio nodes, which are then exposed through the WebAudio API as normal.

Note: You are also able to run WebAssembly inside worklets to handle the processing.

The solution provided above is still useful, as the basic idea holds, but it would ideally use an audio worklet.

Upvotes: 0

padenot
padenot

Reputation: 1555

Use a ScriptProcessorNode, but be aware that if there is too much load on the main thread (the thread that runs your javascript, draws the screen, etc.), it will glitch.

Also, your PCM stream is probably in int16, and the Web Audio API works in terms of float32. Convert it like so:

output_float[i] = (input_int16[i] / 32767);

that is, go from a [0; 65535] range to a [-1.0; 1.0] range.

EDIT I was using output_float[i] = (input_int16[i] / 32767 - 1);, this article shows that you should use output_float[i] = (input_int16[i] / 32767);. Now it's working fine!

Upvotes: 1

Related Questions