How do I stream audio from Pure Data running on a linux web server to the browser?

Question

I'm working on a project where I have a Pd-extended patch (uses lots of externals so I run/patch it in Pd-extended with no extra configuration) that runs an interactive sound art installation. The composer/artist would now like this to run online in a multiuser interactive website. My idea is to:

1. run the Pd-extended patch on a webserver
2. have users interact with the patch (via websockets) (got that working - based on text output from the patch, this is working)
3. stream audio from the patch to the users' browsers

Number 3 is where I'm stuck. I can't get the audio out of PureData into something I can stream to the web browser. I was working with [oggcast~] object yesterday, which uses Icecast2 (http://icecast.org), but it wouldn't connect. I was thinking about sending it to a websocket or from node.js through socket.io or some sort of WebRTC library, but I still need to get it out of PureData.

I found a solution that might work IF I had a sound card to route the audio through. But there is no sound card, since it is just a web server.

It does need to be server side as there are a lot of audio files used and some are very long. Also it would be a huge plus if multiple people could interact and hear each others' interactions. So unfortunately WebPd and any other browser side PD to javascript solution is out.

Any ideas?

Answer 1

I suggest following technique:

1. Generate a virtual alsa device with modprobe snd-aloop

$ sudo modprobe snd-aloop
$ # (assuming you already have two soundcards: hw:0 and hw:1)

2. Prepare loop device (optional)

$ aplay -D hw:2,0,0 Testsong.wav

This will switch the loop device to some setting like Signed 16 bit Little Endian, Rate: 44100 Hz, stereo. This helps pd not to use weired rates.

3. Let pd play on the loop device

Now let pd to output to hw2.

4. Use ffserver to stream from loop device

The second step is to use ffserver (or any other streaming server) and instruct it to use alsa device hw:2,1,0

Answer 2

OK, so I've had plenty of time to research, test and do things. Here is what I found:

libpd and python

Python is a great server side language, with support for websockets and other communication protocols. It also uses libpd - the pure data library/wrapper, available for many programming languages including python. Downside is that it works best for only pd vanilla objects. While I hear that you can add external support, it is not an easy thing to do. I'm currently following this path to a solution...

WebPD

This is NOT a server side solution, so you do not get the benefit of a community/collaborative experience, where everyone can hear at the same time.

The Javascript library that allows you to load a puredata patch and run it via the Web Audio API. Works ok, but a VERY limited amount of objects are supported! Although I was able to get this patch working well using only standard pd vanilla objects, it was still not enough for me to get this patch working with webpd.

icecast streaming

This works great, EXCEPT, there is a LOT of latency. 6-12 seconds! The best we can reduce it to is 6s in the icecast.xml settings below. The rest can only be reduced in the mp3cast~ pd external by changing the source code and recompiling, however buffering/streaming issues may arise... if anyone wishes to try this - please let me know! :)

There is a pure data object (included in pd-extended) called mp3cast~

1. install these packages:

   pd-extended
   

   alsa sound drivers:

   alsa
   alsa-tools
   libasound2-dev
   

   streaming, and dependencies:

   curl
   libxml2
   libxslt
   lame
   icecast2    // This will also launch a setup program in your command line - it will ask you to enter three separate passwords - delete the default (make sure you delete ALL THE WAY to the beginning of the line) then enter your password, each time
   

2. load alsa dummy driver

   Add line to “/etc/modules” (Ubuntu 14.04):

   snd-dummy
   

   Enter this into the Ubuntu command line:

   sudo modprobe snd-dummy            // loads dummy sound card into the kernel
   sudo adduser <yourusername> audio  // adds you to the audio group
   sudo init 6           // reboots ubuntu (any other method of restarting is fine)
   

3. configure icecast2 Edit the file at /etc/icecast2/icecast.xml

   Edit this to cut down latency by 6s: 0 0 Burst on connect pre-buffers so that you don't have any issues on starting up a stream. This is nice, but adds an extra 6s (when is left at the default - 65535).

   Edit this spot with your hostname or ip address and the port that will be typed into the browser (or connected to in your front end code): 111.111.111.111 8000

   And you can also change your passwords, displayed location and contact, mount point (it is /stream by default), etc. But the above is just what NEEDS to be done. Then follow the instructions in the mp3cast~ help file.

4. start it up: launch icecast2 server: sudo /etc/init.d/icecast2 start launch the pd patch: sudo pd-extended -nogui -noadc -rt -alsa /path/to/patch/pdPatch.pd &

   flags: -nogui required! This sets it to NO GUI, because there is none on a server -noadc because we're not receiving audio in, so just don't use it -rt give it realtime priority -alsa force alsa soundcard - probably not necessary, but good practice & - allows you to continue using the command line while pd-extended continues to run