CODEWITHSUNDEEP
javascriptelectronweb-audio-api
Matheus Traldi
Matheus Traldi

Reputation: 27

AudioContext compromising app performance the more audio files I play

I'm developing an Elecron app (JavaScript) to audio visualization. There is a Playlist() instance which receives audio file paths the user wants to play. When the first audio finishes, it plays the next one. So far so good. The app does an intense computational work extracting audio features from each channel, re-rendering canvases and animating plots. It does it beautifully.

The problem is: each time the app plays a next file, the more slow it gets, as if all the audio data before is still somewhere. I've found in documentation the method close() from AudioContext():

"The close() method of the AudioContext Interface closes the audio context, releasing any system audio resources that it uses."

"An AudioContext can now be explicitly closed, thereby releasing any hardware resources associated with the AudioContext. Without this, developers had to depend on garbage collection of the AudioContext to release hardware resources."

I also have found this example of closing and restarting audio contexts:

https://github.com/mdn/webaudio-examples/blob/master/audiocontext-states/index.html

https://mdn.github.io/webaudio-examples/audiocontext-states/

The problem is that I use a audioContext.createMediaElementSource(HTMLelementID) and it doesn't allow me to restart everything recreating all the nodes like in the example. A simplified code that represents what I did before is:

class Audio() {
   constructor(audioElementID, playlistObj) {
      this.audioContext = new AudioContext();
      this.audioElement = document.getElementById(audioElementID);
      this.track = this.audioContext.createMediaElementSource(this.audioElement);
      this.gainNode = this.audioContext.createGain();
      this.track.connect(this.gainNode);
      this.gainNode.connect(this.audioContext.destination);

      this.audioElement.addEventListener('ended', () => {
         playlistObj.playnextTrack() // changes the src from the html element (audioElementID) and sets this.audioElement.currentTime to 0
      }
   }
 // everything is a property here for debugging reasons 
}

const audio = new Audio('audioID', playlist);

// playlist defined somewhere else

To implement the close() method I had to change (just exactly the example, a function that recreates everything again):

class Audio() {
   constructor(audioElementID, playlistObj) {
      this.createAudioContext = () => {
         this.audioContext = new AudioContext();
         this.audioElement = document.getElementById(audioElementID);
         this.track = this.audioContext.createMediaElementSource(this.audioElement);
         this.gainNode = this.audioContext.createGain();
         this.track.connect(this.gainNode);
         this.gainNode.connect(this.audioContext.destination);

         this.audioElement.addEventListener('ended', () => {
            playlistObj.playNextTrack() // changes the src from the html element (audioElementID) and sets this.audioElement.currentTime to 0
         }
      }
      
      this.createAudioContext();
   } 
}

and in playlist.playNextTrack() I pause the audioElement, call audio.audioContext.close(), wait for it (it's a promise), call audio.createAudioContext() to recreate everything and plays. The logic returns an error at this.track = this.audioContext.createMediaElementSource(this.audioElement) with:

"Failed to execute 'createMediaElementSource' on 'BaseAudioContext': HTMLMediaElement already connected previously to a different MediaElementSourceNode, at Audio.createAudioContext"

In the example, the audio source is just a random oscillator and not a mp3 audio file.

I'm really stuck here. Don't know what to do. I'm not even sure if AudioContext() really holds data from all the audio files before causing this performance problem. And if so, how could I reconnect the HTMLMediaElement to a new node audio.createAudioContext() creates? I've already tried audio.track.disconnect()but it doesn't work (as it shouldn't because here I'm disconnecting track from gainNode). And also audioElement doesn't have a disconnect()method as It's just a html element.

Any idea?

UPDATE:

I passed over the problem of recreating the audio context deleting and creating again the html element. But the problem persist: the more new audio files are played, the app gets slower. More precisely now: the more new AudioContext() is created, the slower it gets (even if I close the previous one).

Upvotes: 1

Views: 1246

Answers (1)

Brad
Brad

Reputation: 163538

I'm really stuck here. Don't know what to do. I'm not even sure if AudioContext() really holds data from all the audio files before causing this performance problem.

No, it's really unlikely this is the case. The AudioContext sets up things like the sample rate, output destination, and the graph. That's all.

The close() method of the AudioContext Interface closes the audio context, releasing any system audio resources that it uses.

You're misunderstanding what this means. Those "system audio resources" are the sound devices. While the AudioContext is running, there is an audio device requested. This is particularly meaningful in low power environments, like mobile. Another example would be Bluetooth. If the AudioContext is kept running, your Bluetooth headset may just stay on. If the AudioContext is allowed to close, then the Bluetooth headset may go to sleep.

And if so, how could I recconect the HTMLMediaElement to a new node audio.createAudioContext() creates?

You don't. While it would be nice if the API supported this, it seems it doesn't. Simply create a new HTMLMediaElement.

What you should do is properly profile your application to figure out where the slowdown is occurring. Use your developer tools. Might be faster though just to start commenting out sections of things that are running. We certainly can't tell you where the problem is, specifically, from the code you've shown.

Upvotes: 2

Related Questions