CODEWITHSUNDEEP
audiobinaryweb-audio-apiarraybuffer
Denmark Dionisio
Denmark Dionisio

Reputation: 41

Playing incoming ArrayBuffer audio binary data from binaryjs server simultaneously

Good day! I'm into video chat streaming this morning and I've bumped into a problem with the incoming ArrayBuffer which contains binary data of an audio.

Here is the code I found for playing binary audio data (Uint8Array):

function playByteArray(byteArray) {
    var arrayBuffer = new ArrayBuffer(byteArray.length);
    var bufferView = new Uint8Array(arrayBuffer);
    for (i = 0; i < byteArray.length; i++) {
        bufferView[i] = byteArray[i];
    }
    context.decodeAudioData(arrayBuffer, function(buffer) {
        buf = buffer;
        play();
    });
}

// Play the loaded file
function play() {
    // Create a source node from the buffer
    var source = context.createBufferSource();
    source.buffer = buf;
    // Connect to the final output node (the speakers)
    source.connect(context.destination);
    // Play immediately
    source.start(0);
}

Now below, I've used MediaStreamRecorder from https://github.com/streamproc/MediaStreamRecorder to record the stream from getUserMedia. This code will continuously send the recorded binary data to the server.

if (navigator.getUserMedia) {
        navigator.getUserMedia({audio: true, video: true}, function(stream) {
            video.src = (window.URL || window.webkitURL).createObjectURL(stream); //get this for video strewam url
          video.muted = true;
            multiStreamRecorder = new MultiStreamRecorder(stream);
            multiStreamRecorder.canvas = {
              width: video.width,
              height: video.height
            };
            multiStreamRecorder.video = video;

          multiStreamRecorder.ondataavailable = function(blobs) {
            var audioReader = new FileReader();
              audioReader.addEventListener("loadend", function() {
                var arrBuf = audioReader.result;
                var binary = new Uint8Array(arrBuf);
                streamToServ.write(binary);
                // streamToServ is the binaryjs client
              });
              audioReader.readAsArrayBuffer(blobs.audio);
            };
          multiStreamRecorder.start(1);
        }, onVideoFail);
    } else {
        alert ('failed');
    }

Convert the blobs produced (audio and video) to binary and send it to binaryjs which will be played on another client with this:

client.on('stream', function (stream, meta) {
    stream.on('data', function(data) {
        playByteArray(new Uint8Array(data));
    });
});

I had no problems with transferring the binary data but the problem is there is a hiccup sound in the playback significantly on every binary data that was played. Is there something wrong on how I play the incoming ArrayBuffers? I'm also thinking of asking streamproc about this.

Thanks in advance!

Cheers.

Upvotes: 3

Views: 2501

Answers (2)

Denmark Dionisio
Denmark Dionisio

Reputation: 41

I found a solution to this problem by making an audio buffer queueing. Most of the code is from here:

Choppy/inaudible playback with chunked audio through Web Audio API

Thanks.

Upvotes: 1

Raymond Toy
Raymond Toy

Reputation: 6048

Not sure if this is the problem, but perhaps instead of source.start(0), you should use source.start(time), where time is where you want to start the source. source.start(0) will start playing immediately. If your byte array comes in faster than real-time, the sources might overlap because you start them as soon as possible.

Upvotes: 0

Related Questions