CODEWITHSUNDEEP
audiobluetoothandroid-bluetoothbluetooth-lowenergy
tusharrnimje
tusharrnimje

Reputation: 339

How do voice commands work via Bluetooth remote (Nexus player remote) in Android (Nexus player)?

Can anyone please elaborate following questions?

  1. How bluetooth stack handles audio data?
  2. How audio commands are processed?
  3. Do we need any service to process audio data?

Upvotes: 1

Views: 1628

Answers (2)

bam
bam

Reputation: 194

I just bought some universal Android TV remote and questioned the same thing, as the built-in mic didn't detect in Linux (presumably, it would work on any Android TV box, but my goal was make use of it in Linux).

I found out nowadays all these Android TV remotes are mostly unified regarding built-in mic input architecture, as Google stepped in and produced some Reference design for such remotes specifically for Android TV. That made them incompatible with usual Bluetooth sound input devices (headsets, microphones, etc.), so even plain Android might be out of scope here. That design was cloned by many Chinese vendors since then (with varied grade of success).

As it's turned out, all these built-in mic's now follow so called "Google Voice over BLE spec". This spec apparently is mostly intended for vendors and not publicly available, but I managed to find some essential info from it.

So I created Bluez (Linux Bluetooth stack) support issue and put all related info I could find there. I still didn't find out where on Android TV side things are implemented and if it open sourced at all, so that info is highly welcomed:

https://github.com/bluez/bluez/issues/1086

Upvotes: 0

Nipo
Nipo

Reputation: 2927

Basically, voice commands over BLE require:

  • some audio codec for reducing required bandwidth (ADPCM and SBC are common, OPUS is emerging),
  • some audio streaming method through BLE,
  • decoding and getting the audio stream from BLE daemon to a command processing framework.

In the android world, command processing framework is google sauce (closed) that most easily gets its audio from an ALSA device. What is left to be done is getting audio from the remote to an ALSA device.

So for audio streaming, either you:

  • use a custom L2CAP channel or a custom GATT service, this requires a custom android service app and/or modifications to Bluedroid to handle those, it will need a way to inject audio stream as ALSA, most probably with a "loop" audio device driver,
  • declare audio as custom HID reports, this way, Bluedroid injects them back to the kernel, then add a custom HID driver that processes these reports and exposes an audio device.

Audio over BLE is not standard, so all implementations do not do the actual same thing. In Nexus Player case, implementation uses HID: It streams an ADPCM audio stream, chunked in HID reports. There is a special HID driver "hid-atv-remote.c" in Android linux kernel that exposes an ALSA device in addition to input device. Bluedroid has no information about audio, all it does is forwarding HID reports from BLE to UHID.

Upvotes: 3

Related Questions