Android MediaCodec AAC encoder

Question

I use the MediaCodec class provided by the Android SDK since API level 16 with the OMX.SEC.aac.enc encoder to encode audio to a file. I get the audio input from the AudioRecord class. My instance of the AudioRecord class is configured like this:

bufferSize = AudioRecord.getMinBufferSize(44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_PCM_16BIT);
recorder = new AudioRecord(MediaRecorder.AudioSource.MIC, 44100, AudioFormat.CHANNEL_IN_STEREO, AudioFormat.ENCODING_DEFAULT, bufferSize);

I am able to play the raw data from the AudioRecord instance, so the problem does not reside there.

I write the output from the AudioRecord instance to a ByteBuffer instance and pass it to an available input buffer from the encoder. The output from the encoder is written to a file on the SD-card.

These are the configuration parameters for my MediaCodec instance:

codec = MediaCodec.createEncoderByType("audio/mp4a-latm");
MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
format.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1024);
format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 2);
format.setInteger(MediaFormat.KEY_SAMPLE_RATE, 44100);
format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);
codec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);

VLC tells me that there are no streams in my aac file. The command FFMPEG -i @filename@ gives me the following error: Invalid data found when processing input. None of the mediaplayers I tested are able to play my file.

Why am I unable to play my file? I receive no OpenMAX errors in LogCat and the application does not crash when encoding. I wrote a video encoder that works on the same principle and it works.

This is the code to read the data from the AudioRecord instance to a buffer:

    new Thread() {
        public void run() {
            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(bufferSize);
            int read = 0;
            while (isRecording) {
                read = recorder.read(byteBuffer, bufferSize);
                if(AudioRecord.ERROR_INVALID_OPERATION != read){
                    encoder.add(byteBuffer);
                }
            }
            recorder.stop();
        }
    }.start();

The function add from my encoder copies the content of one buffer to another:

public void add(ByteBuffer input) {
    if (!isRunning)
        return; 

    if (tmpInputBuffer == null)
        tmpInputBuffer = ByteBuffer.allocate(input.capacity());

    if (!tmpBufferClear)
        Log.e("audio encoder", "deadline missed"); //TODO lower bit rate

    synchronized (tmpInputBuffer) {
        tmpInputBuffer.clear();
        tmpInputBuffer.put(input);
        tmpInputBuffer.notifyAll();
        Log.d("audio encoder", "pushed data into tmpInputBuffer");
    }
}

The following code is used to occupy the input buffer of the encoder:

new Thread() {
    public void run() {
        while (isRunning) {
            if (tmpInputBuffer == null)
                continue;
            synchronized (tmpInputBuffer) {
                if (tmpBufferClear) {
                    try {
                        Log.d("audio encoder", "falling asleep");
                        tmpInputBuffer.wait(); //wait when no input is available
                    } 
                    catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }

                ByteBuffer[] inputBuffers = codec.getInputBuffers();
                int inputBufferIndex;
                do
                    inputBufferIndex = codec.dequeueInputBuffer(-1);
                while (inputBufferIndex < 0);
                ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
                inputBuffer.clear();
                Log.d("input buffer size", String.valueOf(inputBuffer.capacity()));
                Log.d("tmp input buffer size", String.valueOf(tmpInputBuffer.capacity()));
                inputBuffer.put(tmpInputBuffer.array());
                tmpInputBuffer.clear();
                codec.queueInputBuffer(inputBufferIndex, 0, tmpInputBuffer.capacity(), 0, 0);
                tmpBufferClear = true;
                Log.d("audio encoder", "added to input buffer");
            }
        }
    }
}.start();

I write the output from the encoder to a local file like this:

    new Thread() {
        public void run() {
            while (isRunning) {
                ByteBuffer[] outputBuffers = codec.getOutputBuffers();
                MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
                int outputBufferIndex = codec.dequeueOutputBuffer(bufferInfo, -1);
                while (outputBufferIndex >= 0) {
                    ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
                    byte[] outData = new byte[bufferInfo.size];
                    outputBuffer.get(outData);

                    try {
                        fileWriter.write(outData, 0, outData.length);
                    } 
                    catch (IOException e) {
                        e.printStackTrace();
                    }
                    codec.releaseOutputBuffer(outputBufferIndex, false);
                    outputBufferIndex = codec.dequeueOutputBuffer(bufferInfo, 0);
                    Log.d("audio encoder", "removed from output buffer");
                }
            }
            codec.stop();

            try {
                fileWriter.close();
            } 
            catch (IOException e) {
                e.printStackTrace();
            }
        }
    }.start();
                tmpBufferClear = true;
                Log.d("audio encoder", "added to input buffer");
            }
        }
    }
}.start();

Answer 1

I had to solve the is use case as well. It is possible, but there are some tricky things to get this working. I've upload my proof of concept app to GitHub. It has a working example of how to do this.

https://github.com/dburckh/EncodeRawAudio

Answer 2

I guess you missed the MediaMuxer class. You need it if you want to write something got from MediaCodec to a file for example.