Receiving corrupted data while sending large data from STM32 to PC over UART

Question

My project is very simple, use a microcontroller to read some analog signal for a certain period then send the data to PC. I am using STM32F756 nucleo board. I have configured one ADC in DMA mode to read the analog signal triggered by a timer (PWM generation). I am using another timer to log the timestamps of the first timer (input capture). Both data are stored in separate arrays in the MCU. The declared array size is 12000 for both as that is the maximum possible data size I am expecting but the actually logged data can be less. For test, I am capturing 6500 data, the rest of the arrays are empty. I am sending the data over usart3 in asynchronous mode. The uart init code generated from CubeMX, discarding comments, is as following.

static void MX_USART3_UART_Init(void)
{
  huart3.Instance = USART3;
  huart3.Init.BaudRate = 115200;
  huart3.Init.WordLength = UART_WORDLENGTH_8B;
  huart3.Init.StopBits = UART_STOPBITS_1;
  huart3.Init.Parity = UART_PARITY_ODD;
  huart3.Init.Mode = UART_MODE_TX_RX;
  huart3.Init.HwFlowCtl = UART_HWCONTROL_NONE;
  huart3.Init.OverSampling = UART_OVERSAMPLING_16;
  huart3.Init.OneBitSampling = UART_ONE_BIT_SAMPLE_DISABLE;
  huart3.AdvancedInit.AdvFeatureInit = UART_ADVFEATURE_NO_INIT;
  if (HAL_UART_Init(&huart3) != HAL_OK)
  {
    Error_Handler();
  }
}

A minimum version of the data sending code is as following.

uint32_t ADC1_Buffer[12000];
uint32_t Timestamp_Buffer[12000];
uint16_t LastSize = 500;
uint8_t Message = "Sending data\r\n";
uint8_t transmitting;
uint8_t RxBuffer;

void HAL_UART_TxCpltCallback(UART_HandleTypeDef *huart)
{
    transmitting = 0;
}

void Send_data(uint8_t* data, uint32_t size)
{
    uint16_t packet_size = 16;
    uint32_t noo_chunks = (size / packet_size);
    uint16_t remainder_size = 0;

    if(size > (noo_chunks * packet_size))
    {
        remainder_size = size - (noo_chunks * packet_size);
    }

    for(uint32_t i = 0; i < noo_chunks; i++)
    {
        HAL_UART_Transmit(&huart3, data+(i*packet_size), packet_size, HAL_MAX_DELAY);
    }

    if(remainder_size)
    {
        HAL_UART_Transmit(&huart3, data+(size-remainder_size), remainder_size, HAL_MAX_DELAY);
    }
}

int main(void)
{
    //Fill up the first 6500 data of both arrays here with anything
    
    //Wait till PC is ready to receive
    while(HAL_UART_Receive(&huart3, &RxBuffer, 1, HAL_MAX_DELAY) != HAL_OK);

    uint16_t DSIZE = 6000 + LastSize;

    HAL_UART_Transmit(&huart3, &Message[0], strlen(Message), 1000);
    HAL_UART_Transmit(&huart3, (uint8_t*)&DSIZE, 2, 1000);

    HAL_UART_Transmit(&huart3, &Message[0], strlen(Message), 1000);

    //HAL_UART_Transmit(&huart3, (uint8_t*)&ADC1_Buffer[0], DSIZE*4, HAL_MAX_DELAY); //Blocking
    //Send_data((uint8_t*)&ADC1_Buffer[0], DSIZE*4); //32 byte packets, no header/trailer
    transmitting = 1; //DMA flag
    HAL_UART_Transmit_DMA(&huart3, (uint8_t*)&ADC1_Buffer[0], DSIZE*4);  //DMA
    while(transmitting);  //DMA flag check

    HAL_UART_Transmit(&huart3, &Message[0], strlen(Message), 1000);

    HAL_Delay(100);

    HAL_UART_Transmit(&huart3, &Message[0], strlen(Message), 1000);

    //HAL_UART_Transmit(&huart3, (uint8_t*)&Timestamp_Buffer[0], DSIZE*4, HAL_MAX_DELAY); //Blocking
    //Send_data((uint8_t*)&Timestamp_Buffer[0], DSIZE*4); //32 byte packets
    transmitting = 1; //DMA flag
    HAL_UART_Transmit_DMA(&huart3, (uint8_t*)&Timestamp_Buffer[0], DSIZE*4); //DMA
    while(transmitting); //DMA flag check

    HAL_UART_Transmit(&huart3, &Message[0], strlen(Message), 1000);

    HAL_UART_Transmit(&huart3, &Message[0], strlen(Message), 1000);

    while(1){}
}

On the receiving PC, I used python to receive the data. The code is as following.

import matplotlib.pyplot as plt
import serial
import struct

com = serial.Serial()
com.port = 'COM4'
com.baudrate = 115200
com.timeout = None
com.bytesize = serial.SEVENBITS
com.parity = serial.PARITY_ODD
com.stopbits = serial.STOPBITS_ONE

trailer = bytes([0x00, 0x00, 0xc0, 0x7f])

print("Start?\n")
input()
txs = bytes([85])

com.open()
if not com.is_open:
    print("Could not connect\n")
else:
    com.write(txs)
    dsize = com.read(2)
    Dsize, = struct.unpack('H', dsize)
    print(Dsize)
    message = com.readline()
    print(message)
    vbuff = com.read(Dsize*4)
    message = com.readline()
    print(message)
    message = com.readline()
    print(message)
    tbuff = com.read(Dsize*4)
    message = com.readline()
    print(message)
    print(len(vbuff))
    print(len(tbuff))
    voltages = list(struct.unpack('{}I'.format(Dsize), vbuff))
    timestamps = list(struct.unpack('{}I'.format(Dsize), tbuff))
    
    plt.plot(timestamps, voltages, 'b-')
    plt.show()

com.close()

with open('Data.csv', 'w') as file:
    for i in range(Dsize):
        file.write("{},{}\n".format(timestamps[i], voltages[i]))

The problem is that the received data seems to have some blocks replaced by another block. For example, the first 31 data comes correctly. Then, the next 32 data gets replaced by the first 31 data (plus the one that should've been in position 32), i.e., the first 32 data are repeated twice. Then, data 64-95 comes correctly. Then the pattern repeats and seems to become more frequent at the later data. Below are some pictures of the received data. The pictures are of data of the timestamps shown in csv format. 3 pairs of columns, using blocking transfer (HAL_UART_Transmit), using manual 32 byte packet transmission (Send_data) and DMA method. For each column pair, the first one is the received timestamp data and the second column is the difference between two consecutive values (data[n] - data[n-1]). I used this and color formatting as a convenient method of finding the problematic data points.

When I first saw the discrepancy in the graph output from python which seemed like the time counting is resuming from zero, my first assumption was that I made a mistake in setting up the timer or handling some interrupt which is causing the timer to be reset before entire data sampling ends. I rechecked the timer configuration and other relevant codes but the timer is not resetting anywhere. It's TIM2 which is 32 bits with ARR set to maximum and running at 4MHz speed. It should not be overflowing before the sampling ends which is at most 1000s long. I checked the Timestamp array in debug mode and indeed the timestamp data in the MCU memory are correct. For example, position 32 has the timestamp 123 128 as expected. But the same data, when sent to PC, has the above mentioned problem. My next guess was that trying to send such large data with one blocking call might be causing the MCU's uart module to malfunction. SO, I tried to send the data in 32 byte (and later 16 byte) packets with blocking calls. This had the same problematic result. Next, I tried to send the data through DMA instead of blocking call but the result was same. In all cases, I am reading the whole data together in PC. My final guess was that it's a problem with python. So, I tried to receive the transmission with PuTTY and later process the log file with simple C code. It still showed the same results.

In the MCU debug mode immediately after transmission ends, I can see that the data are all correct, but in the received data, some parts of the data are missing and replaced by previous block which shows an obvious and repeated pattern. Interestingly, the signal data sampled from ADC is received perfectly without any losses, repetition or any other corruption. Even when verified in the same manner as the timestamp data, the ADC value, for example, of 32nd position is received correctly but the timestamp data is not.

What could be the cause of this data corruption and what can be done to remedy it?

Note: The buffers are filled with DMA in the actual tests. DMA is set in normal mode and all DMA operation stops and both DMA streams are disabled before uart transmission starts.

Note (Edit): For those interested, I also tried to convert the timestamps from timer ticks to second (float value) before sending from MCU. The result was still the same with the same pattern.