Is there a way to stop Web Audio API decodeAudioData method memory leak?

Question

The Problem

When creating audio buffers using the Web Audio API, there are buffers created by the decodeAudioData method, which reside in memory and are apparently not accessible through JavaScript. They seem to hang around for the entire life of a browser tab, and never get garbage collected.

Possible Reason For the Problem

I know that these buffers are separated from the main thread and set on another thread for asynchronous decoding. I also know that the API spec says that decodeAudioData should not be allowed to decode the same input buffer twice, which I assume is why a copy of the decoded buffer and/or the encoded input buffer are kept around. However, on memory limited devices like Chromecast, this causes huge amounts of memory to accumulate and Chromecast crashes.

Reproducibility

In my example code, I fetch an mp3 using Ajax and then pass the arraybuffer into the decodeAudioData function. Normally within that function there is a onsuccess callback which can take the decoded AudioBuffer as a parameter. But here in my code, I don't even pass that in. Therefore I also don't do anything with the decoded buffer after decoding it. It is not referenced anywhere within my code. It is entirely left in the native code. However, every call to this function increases the memory allocation and it is never released. For example, in Firefox about:memory shows the audiobuffers there for the life of the Tab. Non-reference should be sufficient for the garbage collector to get rid of these buffers.

My main question then is, is there any reference to these decoded audio buffers, say within the audiocontext object, or somewhere else that I can try to remove them from memory? Or is there any other way that I can cause these stored and unreachable buffers to disappear?

My question differs from all the others currently on SO regarding decodeAudioData because I show that the memory leak happens even without the user storing any reference or even using the returned decoded audio buffer.

Code To Reproduce

function loadBuffer() {
    // create an audio context
    var context = new (window.AudioContext || window.webkitAudioContext)();

    // fetch mp3 as an arraybuffer async
    var url = "beep.mp3";
    var request = new XMLHttpRequest();
    request.open("GET", url, true);
    request.responseType = "arraybuffer";

    request.onload = function () {

        context.decodeAudioData(
                request.response,
                function () {// not even passing buffer into this function as a parameter
                    console.log("just got tiny beep file and did nothing with it, and yet there are audio buffers in memory that never seem to be released or gc'd");
                },
                function (error) {
                    console.error('decodeAudioData error', error);
                }
        );
    };

    request.onerror = function () {
        console.log('error loading mp3');
    }
    request.send();
}

To anticipate some possible responses.

1. I must use Web Audio API because I am playing four part harmony from four audio files on Chromecast and the html audio element does not support multiple simultaneous playback on Chromecast.
2. Probably any JS library you may reference [e.g. Howler.js, Tone.js, Amplitude.js etc.] is built upon the Web Audio API, and so they will all share this memory leak problem.
3. I know that the WAA is implementation dependent on a per browser basis. My primary concern at the moment is Chromecast, but the problem exists for every browser I've tried.
4. Therefore, I think it is a spec related issue where the spec requires the non-dupe encoding rule, and so implementers keep copies of the buffer around on a browser level thread so they can check them against new xhr inputs. If the spec writer's happen to read my question, is there not a way that the user can have the option for this behavior, and opt out of it if they wish in order to prevent the internal buffer storage on mobile and thin memory platforms?
5. I have not been able to find any reference to these buffers in any JS object.
6. I know that I can audio_context.close() and then hope for garbage collection of all the resources held by the audio_context, and then hope that I can reinstantiate the audio_context with a new one, but that has not empirically been timely enough for my application. Chromecast crashes before GC takes out the trash.

Answer 1

A lot of answers I have seen seem to overcomplicate this. I have run into this same issue while rebuilding an audio system for an application I'm building, but then I realised it previously was not an issue, this was because everytime I played a new audio I closed the previous AudioContext and used the variable it was referenced in for a new AudioContext.
This means that the only two things one has to do to clear this overly memory usage is to use AudioContext.close(), and remove references to it, disconnecting nodes and such is not required.

Answer 2

I was facing the same problem. What eventually worked for me was to disconnected and delete all connected resources:

    if (this.source) {
      this.source.disconnect()
      delete this.source
    }

    if (this.gain) {
      this.gain.disconnect()
      delete this.gain
    }

    await this.audioContext.close()

    delete this.audioContext
    delete this.audioBuffer

Just closing the audioContext is not enough. It seems that references will continue to exist preventing garbage collection.

Answer 3

Pragmatic Workaround

I have found a method to solve the problem of the Web Audio API audiobuffers handing around indefinitely and crashing Chromecast and other mobile platforms. [[ I have not tested this on all browsers - your mileage may vary. ]]

LOADING STAGE

1. Load the document using Web Audio API inside an iFrame.
2. Load your audio buffers and do whatever you do to play them.

CLEARING STAGE

3. Call sourceNode.stop on all of the playing nodes you have reference to.
4. Call source.disconnect(); on all source nodes.
5. Call gainNode.disconnect(); on all gain nodes those source nodes are associated with (and whatever other kind of WAA nodes you might be using that have a disconnect method)
6. Set all referenced gainNodes and sourceNodes to null;
7. Null out any buffers you have referenced both decoded and your xhr fetched encoded audiobuffers;
8. KEY: Within the WAA page call audio_context.close(); then set audio_context=null; (this can be done from the parent of the iFrame using contentWindow).
9. Note: Some of these nulling steps may not be absolutely necessary, however this approach has worked for me.

RE-LOADING STAGE

10. Reload the iframe from the parent page. This will cause all of the audiobuffers to be garbage collected ON THE NEXT GC ROUND, including the ones in the hidden (non JS) areas of memory.
11. Your iframe will have to reinstantiate the web audio context and load its buffers and create nodes etc. just as you did when you first loaded it.

Notes: You must decide when you are going to use this clearing method (e.g. after so many buffers have been loaded and played). You can do it without an iframe, but you may have to reload the page once or twice to get garbage collection to fire. This is a pragmatic workaround for those who need to load lots of Web Audio API audio buffers on memory thin platforms like Chromecast or other mobile devices.

FROM PARENT

  function hack_memory_management() {
                var frame_player = document.getElementById("castFrame");
                //sample is the object which holds an audio_context
               frame_player.contentWindow.sample.clearBuffers();
                 setTimeout(function () {
                    frame_player.contentWindow.location.reload();
                }, 1000);
            }

INSIDE WAA IFRAME

CrossfadeSample.prototype.clearBuffers = function () {
    console.log("CLEARING ALL BUFFERS -IT'S UP TO GC NOW'");
    // I have four of each thing because I am doing four part harmony

    // these are the decoded audiobuffers used to be passed to the source nodes
    this.soprano = null;
    this.alto = null;
    this.tenor = null;
    this.bass = null;
    if (this.ctl1) {

        //these are the control handles which hold a source node and gain node 
        var offName = 'stop';
        this.ctl1.source[offName](0);
        this.ctl2.source[offName](0);
        this.ctl3.source[offName](0);
        this.ctl4.source[offName](0);

        // MAX GARGABE COLLECTION PARANOIA

        //disconnect all source nodes
        this.ctl1.source.disconnect();
        this.ctl2.source.disconnect();
        this.ctl3.source.disconnect();
        this.ctl4.source.disconnect();

        //disconnect all gain nodes
        this.ctl1.gainNode.disconnect();
        this.ctl2.gainNode.disconnect();
        this.ctl3.gainNode.disconnect();
        this.ctl4.gainNode.disconnect();

        // null out all source and gain nodes
        this.ctl1.source = null;
        this.ctl2.source = null;
        this.ctl3.source = null;
        this.ctl4.source = null;

        this.ctl1.gainNode = null;
        this.ctl2.gainNode = null;
        this.ctl3.gainNode = null;
        this.ctl4.gainNode = null;
    }

    // null out the controls
    this.ctl1 = null;
    this.ctl2 = null;
    this.ctl3 = null;
    this.ctl4 = null;

    // close the audio context
    if (this.audio_context) {
        this.audio_context.close();
    }
    // null the audio context
    this.audio_context = null;

};

Update:

Sadly, even this does not reliably work and Chromecast can still crash given a few clear and loads of new mp3s. See "My present solution" elsewhere on this page.

Answer 4

My present solution

I could not find a final satisfactory solution for Chromecast using the Web Audio API and simultaneous playback of four mp3s - used for four part harmony. The 2nd Gen seems to simply not have enough resources to hold the audiobuffers and simultaneously decode four mp3 files using decodeAudioData without leaving too much garbage around and eventually crashing. I decided to go with surikov's webaudiofont which is built on top of the Web Audio API, and to use midi files. I never had a problem on desktop browsers or other devices with more resources, but I have to have it work on Chromecast. I have no problems at all now using webaudiofont.

Answer 5

Can you maybe use multiple audio-tags on Chromecast when you route each of them into the Web Audio graph (by using a MediaElementAudioSourceNode)?