CODEWITHSUNDEEP
delphiweb-audio-apihowler.jstone.jstms-web-core
David Schwartz
David Schwartz

Reputation: 1838

Using Web Audio API to work with both short and long audio samples

I have a situation where I'm using the Web Audio API (WAA) to play a lot of short MP3 files in sequence, in one track (typically 5-15 seconds in length), and a couple of other tracks where I'll loop a single 2-5 minute audio file. The first track is working great. I'm tackling the other two tracks now, and the WAA spec says the buffers are limited to 60-seconds of audio. So I'm trying to figure out the best way to handle the other tracks with longer samples.

I've found a bunch of posts loosely relating to this topic, but most are quite old and were written before WAA V1.0 spec came out. A lot of them recommend using the HTML5 <AUDIO> elements to play longer files, but they don't seem to offer the same control as I'm getting by managing a series of buffers with shorter audio samples in them. However, I have not found anything that fits what I'm trying to do here: combining a track made up of a lot of short samples with one or more tracks that each have one longer sample (>60 secs) -- they seem to deal with one or the other, but not both.

I know there are a couple of other libraries that let you work with longer audio samples, like Howler.js and Tone.js. They're supposedly built on top of WAA, but it's not clear if they can be used to manage separate tracks connected to the same WAA audio context.

Alternatively, can I use them with a separate audio context that runs in parallel with the first? (This might be the best approach if it works.)

I've seen suggestions about splitting up the longer files into shorter ones, but you can't just chop up an MP3 file at arbitrary points, right? I'd think it needs to be loaded into a buffer and decoded into a WAV or PCM format first. If someone is able to say, "I want to play this MP3 file here" in my web app and it's 5 minutes long, then how would that be done if the buffers can't hold files that big? (I know some browsers can, but I need to deal with the broader case and stick to the limits that I know ALL browsers should be able to handle.)

I'm looking for suggestions on what might be the best way(s) to manage longer audio samples in separate tracks that will be playing in sync with the first track.

Note that timing between tracks is not critical. All of the tracks will start playing together. They'll play until the first one has played all of its contents, then they'll stop. If it goes longer than the others, the others will just loop.

In case it matters, I'm working with Delphi and the TMS WEB Core platform that translates the Delphi code into javascript. There's a unit that lets me access WAA and it works great. I can also inject javascript into the code where needed, which is what I'd need to do to use Howler or Tone.

Upvotes: 1

Views: 512

Answers (2)

David Schwartz
David Schwartz

Reputation: 1838

Here is the solution I found using ffmpeg:

ffmpeg -i mysong.mp3 -f segment -segment_time 30 -c copy mysong_%03d.mp3

Let's break down the command:

  • ffmpeg invokes the ffmpeg command-line tool.
  • -i mysong.mp3 specifies the input file (mysong.mp3 in this case).
  • -f segment tells ffmpeg to use the segment muxer for splitting the file.
  • -segment_time 30 sets the duration of each segment to 30 seconds.
  • -c copy enables stream copy mode, which reuses the existing audio and video codecs without re-encoding them.
  • output%03d.mp3 specifies the output file names. %03d creates a sequence of numbered files starting from 001.

After executing this command, ffmpeg will split the mysong.mp3 file into multiple segments, each 30 seconds long, with filenames like output001.mp3, output002.mp3, and so on.

Upvotes: 1

chrisguttandin
chrisguttandin

Reputation: 9076

I'm not sure where the 60 seconds limit is coming from but it's definitely only a recommendation and not a hard requirement. The real limit is defined by the browser/operating system/device combination. Creating a new AudioBuffer is supposed to work as long as there is enough memory available no matter how long it is.

But you can use some tricks to reduce the amount of memory needed for keeping your files in memory. You could for example decode them at a lower sample rate. It would require the Web Audio API to resample them on the fly but it would require less memory to keep the decoded files in memory.

Or you could indeed decode your MP3s partially. It should work with WebCodecs one day but until then you could use a similar technique like phonograph. It's scanning MP3s for frame boundaries in order to cut them exactly at those points.

Upvotes: 0

Related Questions