how can I send packets of audio data using ESP Now between two ESP32 MCUs?

Question

I connected a microphone and speaker to 2 ESP32s and was able to hear my voice reproduced by the speaker from the microphone using an ESP32. I was also able to send audio signals between them, but I was not able to comprehend the spoken words. I assume it is because I am sending the current value from the analogRead function and since the data rate of the ESP Now link is lower than the adc sampling rate, much information is lost in the transmission.

I tried creating an array of 1000 values and sending one array at a time using the code below but its not working for me. I will appreciate your help, I need to send higher quality audio between the 2 ESP32s.

#include <esp_now.h>
#include <WiFi.h>

// REPLACE WITH THE MAC Address of your receiver
uint8_t broadcastAddress[] = {0x7C, 0x9E, 0xBD, 0x47, 0x92, 0x4C}; //MAC Address

float A;
const int Max = 1000;

//Define Sent Message
typedef struct test_message {
  float A[Max];
} test_message;

test_message tx; //Sent Message
test_message rx; // Received Message

// Transmitting Callback
void OnDataSent(const uint8_t *mac_addr, esp_now_send_status_t status) {
  char macStr[18];
  // Copies the sender mac address to a string
  snprintf(macStr, sizeof(macStr), "%02x:%02x:%02x:%02x:%02x:%02x:", mac_addr[0], mac_addr[1], mac_addr[2], mac_addr[3], mac_addr[4], mac_addr[5]);
}

// Receiving Callback
void OnDataRecv(const uint8_t * mac, const uint8_t *incomingData, int len) {
  memcpy(&rx, incomingData, sizeof(rx));
}

void setup() {
  Serial.begin(115200);
  // Set device as a Wi-Fi Station
  WiFi.mode(WIFI_STA);
  // Init ESP-NOW
  if (esp_now_init() != ESP_OK) {
   // Serial.println("Error initializing ESP-NOW");
    return;
  esp_now_register_recv_cb(OnDataRecv);
  }
  // Once ESPNow is successfully Init, we will register for Send CB to
  // get the status of Trasnmitted packet
  esp_now_register_send_cb(OnDataSent);
  
  // Register peer
  esp_now_peer_info_t peerInfo;
  memcpy(peerInfo.peer_addr, broadcastAddress, 6);
  peerInfo.channel = 0;
  peerInfo.encrypt = false;
  
  // Add peer        
  if (esp_now_add_peer(&peerInfo) != ESP_OK){
   // Serial.println("Failed to add peer");
    return;
  }
  // Register for a callback function that will be called when data is received
  esp_now_register_recv_cb(OnDataRecv);
}

void loop() {
  for (int i=0; i<Max; i++){
  tx.A[i] = analogRead(35);
  }
  esp_err_t result = esp_now_send(0, (uint8_t *) &tx, sizeof(test_message));
  for (int i=0; i<Max; i++){
  dacWrite(25, rx.A[i]);
  Serial.println(rx.A[i]);
  }
}

Answer 1

There are a number of issues.

---

First stage of input stores garbage values

In OnDataRecv, you are using memcpy to copy from a uint8_t array into a float array. That won't work. It is UB (undefined behavior) and bad "type punning".

This just puts "garbage" values into the output buffer.

Ditch the float [everywhere] in favor of uint8_t. Your ADC is [probably] only 8 bits, so doing float won't help and is actually less accurate because not all 8 bit values have integral float values.

---

First stage input blindly copies a fixed amount of data

In OnDataRecv, you are ignoring the value of len and blindly copying 1000 [float] samples. You are appending garbage values to the output buffer.

You need to honor partial transfers. For example, on successive calls to OnDataRecv, the len parameter will probably vary a bit. For example, it may be 50 on one call and 23 on the next call, etc.

You probably need a ring queue to accumulate the data.

---

When sending ...

When you send the output buffer to the remote system, you are [blindly] sending 1000 samples.

Again, you must honor partial transfers.

Your ADC [probably] has a sample rate of 48K [8 bit samples] (e.g.). But, your serial port [at 110,000 baud] can only send ~11,000 bytes/second.

So, you need to downsample from 48,000 bytes to 10,000 bytes or so.

A simple way is to only send every fourth byte. Or, you could take the average of each raw four byte sequence.

---

Some compression may be in order ...

You could start by copying to another buffer, the difference of successive samples. Using the difference may make the resulting buffer more compressible.

Then, you could compress the difference buffer using LZW compression. Or, use an arithmetic compression scheme like adaptive Huffman, or CABAC, or some other adaptive [standard] audio compression scheme.

Also, the human ear has a logarithmic response. So, an alternative is to do what .wav files can do [and what the telephone company used to do ;-)] and send 8 bit mulaw or A-law samples

---

Anything you do on the sender has to be reversed on the receiver

So, since you probably want to send variable length packets, you'll need a struct that you send before the data that describes the length of the audio data, and whatever compression scheme was used for the given packet.

I'd add some timestamps as well, so the receiver can output the precise number of samples to maintain the exact output frequency.

And, I might add a CRC, so the receiver can reject/ignore any corrupted samples (replacing them with silence). This is particularly important if sending compressed data as a corruption in that can have disastrous consequences.

The timestamps can allow the receiver to maintain sync even if there is missing/corrupted data.

---

UPDATE:

Another issue is that ESP-NOW only allows 250 bytes as a payload, so sending 1000 of anything other than bits is just going to get truncated. And what's the point of copying the sending MAC address into a local variable in the transmit callback and then just returning? – romkey

Both good points and they got me thinking ...

Using Serial.println sends [EOS terminated] string data. It is not suitable for sending binary data.

To send binary data, we want to use [outside the loop]:

Serial.write(&rx.A[0],Alen);

Where Alen is the actual number of bytes in the buffer. And [per Romkey], we may need to cap it at 250.

The best way is to utilize the return value of write and loop on partial transfers:

int off = 0;
int xlen;

for (;  Alen > 0;  Alen -= xlen, off += xlen) {
    xlen = Alen;

    if (xlen > 250)
        xlen = 250;

    xlen = Serial.write(&rx.A[off],xlen);
}