CODEWITHSUNDEEP
javaapiaudiosignal-processing
Jeremias Araujo
Jeremias Araujo

Reputation: 189

Audio samples mixing or changing volume causes saturation and white noise

I have a multichannel input (i'm using Soundflower 64ch on mac), and I'm trying to mixdown 4 channels of the 64 channels to an stereo output.

What i am doing is, reading chunks of 1024 frames, with 64 channels every frame, then converting the bytebuffer to Short array (values between -32,768 <-> 32,767, because samples are 16 bits).

This way I add for example channel1[sample] + channel2[sample] and I get the mix of both channels. But here is a problem, the sum can overflow the Short (16 bit) range, introducing saturation in the sound. So what I'm doing is (channel1[sample] + channel2[sample]) / 2 but when I divide by 2, I hear a lot of white sound.

Also if I try to reduce the volumen of a channel by doing channel1[sample] * 0.5 there is a lot of saturation. Why does it happen?

Here is my full code, note that I'm converting bytes to short to handle better, and then I'm converting back to bytes for write the mix to the stereo output:

public static void main(String[] args) throws LineUnavailableException {

    int inputChannels = 64;

    AudioFormat inputFormat = new AudioFormat(48000, 16, inputChannels, true, false);
    AudioFormat outputFormat = new AudioFormat(48000, 16, 2, true, false);

    TargetDataLine mic = AudioSystem.getTargetDataLine(inputFormat);
    SourceDataLine speaker = AudioSystem.getSourceDataLine(outputFormat);

    mic.open(inputFormat);
    speaker.open(outputFormat);
    mic.start();
    speaker.start();


    AudioInputStream audioInputStream = new AudioInputStream(mic);

    int bytesPerFrame = audioInputStream.getFormat().getFrameSize();

    // Set an arbitrary buffer size of 1024 frames.
    int CHUNK = 1024 ;
    int numBytes = CHUNK * bytesPerFrame;
    byte[] audioBytes = new byte[numBytes];

    try {
        byte[][] frames = new byte[CHUNK][bytesPerFrame];
        int i = 0, j = 0
                ;
        while (true) {
            // read to audioBytes.
            audioInputStream.read(audioBytes);

            // split audioBytes in _CHUNK_ frames (1024 frames)
            for(j=0; j<CHUNK; j++) {
                frames[j] = Arrays.copyOfRange(audioBytes, j * bytesPerFrame, j * bytesPerFrame + bytesPerFrame);
            }

            // convert bytearray to shortarray
            short[][] shortFrames = new short[CHUNK][inputChannels];
            for(i=0; i < frames.length; i++) {
                ByteBuffer.wrap(frames[i]).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(shortFrames[i]);
            }

            short[] leftOutput = new short[CHUNK*2];
            short[] rightOutput = new short[CHUNK*2];

            for (i=0; i<CHUNK; i++) {
                short channel1 = shortFrames[i][0];
                short channel2 = shortFrames[i][1];
                short channel3 = shortFrames[i][2];
                short channel4 = shortFrames[i][3];

                leftOutput[i] = (short)(channel4);
                rightOutput[i] = (short)(channel4);;
            }


            //convert shortarray in byte buffer
            ByteBuffer byteBuf = ByteBuffer.allocate(CHUNK * 2 * 2); // 2 bytes * 2 output channels
            for (i=0; i<CHUNK; i++) {

                byteBuf.putShort(leftOutput[i]);
                byteBuf.putShort(rightOutput[i]);
            }

            speaker.write(byteBuf.array(),0,byteBuf.array().length);

        }
    } catch (Exception ex) {
        // Handle the error...
        System.out.println("exception");
        System.out.println(ex.toString());
    }
}

Upvotes: 0

Views: 72

Answers (1)

Phil Freihofner
Phil Freihofner

Reputation: 7910

IDK if the issue is how the bytes are being converted to shorts and back, but since you asked about this in the comment, I will post it. Assume buffer has contiguous little-endian bytes at 16-bit encoding. Just reverse the byte indexes for big-endian.

pcmShort = ( buffer[i] & 0xff ) | ( buffer[i+1] << 8 );

The conversion of pcm to byte that I use follows (for little-endian, reverse the indexes for big-endian):

outBuffer[i] = (byte)pcmShort[0];
outBuffer[i+1] = (byte)((int)pcmShort[0] >> 8);

Maybe you can use the two methods (your attempt with ByteBuffer and getShort, and the above) side-by-side on the same data and check if the resulting arrays hold the same values?

Another thing I'd try to do is to just get a single track working. If that sounds okay, then check on the mixing. It's kind of unlikely that the signals are so hot that they are overrunning. So something else is probably going on.

I should try this out myself, I'm not sure when I'll get to it. It could potentially be an improvement over what I've been doing.

Upvotes: 0

Related Questions