How to record audio on Android Q using PlaybackCapture APIs?

Question

I am trying to record audio on Android 10(Q) using Playback Capture API. As Playback Capture API, only allow to record sound with USAGE_GAME, USAGE_MEDIA or USAGE_UNKNOWN, so, I have downloaded Uamp sample that has USAGE_MEDIA set when playing songs. I have also added android:allowAudioPlaybackCapture="true" in the AndroidManifest.xml. Then I have launched the Uamp, start playing the song and keep it in the background.

I have developed a CaptureAudio project with targetSdk 29 and installed it on my OnePlus 7 Pro which has Android 10 installed. I have two buttons on the UI for start and stop the capture. When the application starts capture then read function fills all 0's in the buffer.

To use Playback Capture in the project, I have set up it as follows:

1. Manifest:

        <?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    package="com.example.captureaudio">

    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.CAPTURE_AUDIO_OUTPUT" />
    <uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

    <application
        android:allowBackup="false"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme"
        tools:ignore="GoogleAppIndexingWarning">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>

        <service
            android:name=".services.MediaProjectionService"
            android:enabled="true"
            android:exported="false"
            android:foregroundServiceType="mediaProjection"
            tools:targetApi="q" />
    </application>

</manifest>

2. MainActivity:

class MainActivity : AppCompatActivity() {

    companion object {
        private const val REQUEST_CODE_CAPTURE_INTENT = 1
        private const val TAG = "CaptureAudio"
        private const val RECORDER_SAMPLE_RATE = 48000
        private const val RECORDER_CHANNELS = AudioFormat.CHANNEL_IN_MONO
        //or AudioFormat.CHANNEL_IN_BACK
        private const val RECORDER_AUDIO_ENCODING = AudioFormat.ENCODING_PCM_16BIT
        //  AudioFormat.ENCODING_PCM_16BIT
    }

    private var audioRecord: AudioRecord? = null
    private val mediaProjectionManager by lazy { (this@MainActivity).getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager }
    private val rxPermissions by lazy { RxPermissions(this) }
    private val minBufferSize by lazy {
        AudioRecord.getMinBufferSize(
            RECORDER_SAMPLE_RATE,
            RECORDER_CHANNELS, RECORDER_AUDIO_ENCODING
        )
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        val intent = Intent(this, MediaProjectionService::class.java)
        startForegroundService(intent)
        getPermissions()
    }

    private fun getPermissions() {
        rxPermissions
            .request(
                Manifest.permission.RECORD_AUDIO,
                Manifest.permission.FOREGROUND_SERVICE,
                Manifest.permission.WRITE_EXTERNAL_STORAGE
            )
            .subscribe {
                log("Permission result: $it")
                if (it) { // Always true pre-M
                    val captureIntent = mediaProjectionManager.createScreenCaptureIntent()
                    startActivityForResult(captureIntent, REQUEST_CODE_CAPTURE_INTENT)
                } else {
                    getPermissions()
                }
            }
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        if (requestCode == REQUEST_CODE_CAPTURE_INTENT && data != null) {
            val mediaProjection = mediaProjectionManager.getMediaProjection(resultCode, data)
            val playbackConfig = AudioPlaybackCaptureConfiguration.Builder(mediaProjection)
                .addMatchingUsage(AudioAttributes.USAGE_MEDIA)
                .addMatchingUsage(AudioAttributes.USAGE_UNKNOWN)
                .addMatchingUsage(AudioAttributes.USAGE_GAME)
                .build()
            audioRecord = AudioRecord.Builder()
                .setAudioPlaybackCaptureConfig(playbackConfig)
                .setBufferSizeInBytes(minBufferSize * 2)
                .setAudioFormat(
                    AudioFormat.Builder()
                        .setEncoding(RECORDER_AUDIO_ENCODING)
                        .setSampleRate(RECORDER_SAMPLE_RATE)
                        .setChannelMask(RECORDER_CHANNELS)
                        .build()
                )
                .build()
        }
    }

    fun startCapture(view: View) {
        audioRecord?.apply {
            startRecording()
            log("Is stopped: $state $recordingState")
            startRecordingIntoFile()
        }
        stopRecBtn.visibility = View.VISIBLE
        startRecBtn.visibility = View.INVISIBLE
    }

    private fun AudioRecord.startRecordingIntoFile() {
        val file = File(
            getExternalFilesDir(Environment.DIRECTORY_MUSIC),
            "temp.wav"
            //System.currentTimeMillis().toString() + ".wav"
        )
        if (!file.exists())
            file.createNewFile()

        GlobalScope.launch {
            val out = file.outputStream()
            audioRecord.apply {
                while (recordingState == AudioRecord.RECORDSTATE_RECORDING) {

                    val buffer = ShortArray(minBufferSize)//ByteBuffer.allocate(MIN_BUFFER_SIZE)
                    val result = read(buffer, 0, minBufferSize)

                    // Checking if I am actually getting something in a buffer
                    val b: Short = 0
                    var nonZeroValueCount = 0
                    for (i in 0 until minBufferSize) {
                        if (buffer[i] != b) {
                            nonZeroValueCount += 1
                            log("Value: ${buffer[i]}")
                        }
                    }
                    if (nonZeroValueCount != 0) {

                        // Record the non-zero values in the file..
                        log("Result $nonZeroValueCount")
                        when (result) {
                            AudioRecord.ERROR -> showToast("ERROR")
                            AudioRecord.ERROR_INVALID_OPERATION -> showToast("ERROR_INVALID_OPERATION")
                            AudioRecord.ERROR_DEAD_OBJECT -> showToast("ERROR_DEAD_OBJECT")
                            AudioRecord.ERROR_BAD_VALUE -> showToast("ERROR_BAD_VALUE")
                            else -> {
                                log("Appending $buffer into ${file.absolutePath}")
                                out.write(shortToByte(buffer))
                            }
                        }
                    }
                }
            }
            out.close()
        }
    }

    private fun shortToByte(shortArray: ShortArray): ByteArray {
        val byteOut = ByteArray(shortArray.size * 2)
        ByteBuffer.wrap(byteOut).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().put(shortArray)
        return byteOut
    }

    private fun showToast(msg: String) {
        runOnUiThread {
            log("Toast: $msg")
            Toast.makeText(this@MainActivity, msg, Toast.LENGTH_LONG).show()
        }
    }

    fun stopCapture(view: View) {
        audioRecord?.apply {
            stop()
            log("Is stopped: $state $recordingState")
        }
        stopRecBtn.visibility = View.INVISIBLE
        startRecBtn.visibility = View.VISIBLE
    }

    private fun log(msg: String) {
        Log.d(TAG, msg)
    }

    override fun onDestroy() {
        super.onDestroy()
        audioRecord?.stop()
        audioRecord?.release()
        audioRecord = null
    }
}

3. MediaProjectionService

    class MediaProjectionService : Service() {

    companion object {
        private const val CHANNEL_ID = "ForegroundServiceChannel"
    }

    override fun onBind(intent: Intent?): IBinder? {
        return null
    }

    override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {

        createNotificationChannel()
        val notificationIntent = Intent(this, MainActivity::class.java)
        val pendingIntent = PendingIntent.getActivity(
            this,
            0, notificationIntent, 0
        )

        val notification = NotificationCompat.Builder(this, CHANNEL_ID)
            .setContentTitle("Foreground Service")
            .setContentText("Call Recording Service")
//            .setSmallIcon(R.drawable.ic_stat_name)
            .setContentIntent(pendingIntent)
            .build()

        startForeground(1, notification)
        return START_NOT_STICKY
    }

    private fun createNotificationChannel() {
        val serviceChannel = NotificationChannel(
            CHANNEL_ID,
            "Foreground Service Channel",
            NotificationManager.IMPORTANCE_DEFAULT
        )

        val manager = getSystemService(NotificationManager::class.java)
        manager!!.createNotificationChannel(serviceChannel)
    }
}

The problem is,

1. File /storage/emulated/0/Android/data/com.example.captureaudio/files/Music/temp.wav created but it has only 0s inside it. I have also checked it with xxd /storage/emulated/0/Android/data/com.example.captureaudio/files/Music/temp.wav as following:

OnePlus7Pro:/sdcard # xxd /storage/emulated/0/Android/data/com.example.captureaudio/files/Music/temp.wav | head
00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................

2. On a play from the device, it gives an error "Couldn't play the track you requested".

Any help or suggestion, what I am missing?

Answer 1

I think that something went wrong when you write audio data to your .wav file.

Here is my example app that record audio on Android 10(Q) using Playback Capture API. In this app, I write audio data to .pcm file and then decode it to .mp3 audio file which you can listen to and execute with any player.

Warning!

QRecorder app implements lib lame with NDK.

If you don't want to waste time with importing lib lame to your project, you can decode recorded .pcm file with PCM-Decoder library