CODEWITHSUNDEEP
javascriptpythonweb-audio-apiopusweb-mediarecorder
roman-vygon
roman-vygon

Reputation: 23

Convert int8 opus-encoded array into raw waveform (Python)

I have a js script that records audio from microphone and sends it via a websocket to a python REST API.

Part of js script (works in console):

const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
var options = {mimeType: "audio/webm;codecs=opus", audioBitsPerSecond:16000};
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.addEventListener("dataavailable", function(event) {
    var reader = new FileReader();
    reader.addEventListener("loadend", function() {
        var int8View = new Int8Array(reader.result);
        console.log(int8View);
    });
    reader.readAsArrayBuffer(event.data);
    
});
mediaRecorder.start(200);

If I understand correctly the int8array that this code outputs is opus-encoded audio.

How can I decode it into a raw waveform (something that soundfile.read would return, for example) using python?

I can use ffmpeg to decode opus, but it works on files, how can I turn this array into a proper .opus or .ogg file on python side (since I don't want to send files via a websocket)?

Upvotes: 1

Views: 650

Answers (1)

anthumchris
anthumchris

Reputation: 9090

how can I turn this array into a proper .opus or .ogg file?

You're capturing a Webm file stream via audio/webm;codecs=opus, so that would need to be converted from Webm to Ogg Opus. Instead, you could simply download or send the Webm Opus stream as a file as-is:

const recordedBuffers = []

reader.addEventListener('loadend', () => {
  recordedBuffers.push(reader.result)
})

function download() {
  const a = document.createElement('a')
  a.href = URL.createObjectURL(new Blob(recordedBuffers))
  a.download = 'my-recorded-audio.webm'
  a.click()
}

function saveToServer(url) {
  return fetch(url, {
    method: 'post',
    body: new Blob(recordedBuffers),
    headers: {
      'content-type': 'application/octet-stream',  // optional
      'content-type': 'audio/webm',                // optional
    },
  })
}

Upvotes: 1

Related Questions