How to correctly parse received serial data into lines?

Question

I'm creating a program which communicates with a serial device which is constantly sending data. I'm reading data from device every 100ms (using a timer). I use port.ReadExisting() to receive all currently available data from the device then I try split it into lines, because I need to check some of the received data and the best way is to check lines. The problem occurs when device sends data which doesn't end with "\r\n" or '\n'.

In a perfect situation port.ReadExisting() returns: "sampletext\r\nsomesampletext\nsampletext\r\n

But a problem occurs when there's no CR or LF character at the end:

First time port.ReadExisting() returns this: "text\nsamp"

Second time port.ReadExisting() returns this: letext\r\ntext\r\n"

End result should look like this:

text
sampletext
text

But what I get looks like this:

text
samp
letext
text

My code:

This is the timer which runs every 100ms:

private void CommandTimer_Tick(object sender, EventArgs e)
{
    BackgroundWorker seriaDataWorker = new BackgroundWorker();
    seriaDataWorker.DoWork += (obj, p) => PrintSerialData();
    seriaDataWorker.RunWorkerAsync();
}

BackgroundWorker which gets called by the timer:

private void PrintSerialData()
    {
        try
        {
            if (RandomReboot)
            {
                RebootWatch.Start();
            }
            if (COMport.IsOpen)
            {
                if (COMport.BytesToRead != 0)
                {
                    SerialPrint(COMport.ReadExisting());
                }
            }
        }
        catch (System.IO.IOException SerialException)
        {
            return;
        }
        
    }

Function which parses received data into lines:

private void SerialPrint(string data)
    {
        using (var buffer = new StringReader(data))
        {
            string line = "";


            
            while((line = buffer.ReadLine()) != null)
            {


                if (CheckForAnsw)
                {
                    ReceivedCommandData = line;
                    if (ReceivedCommandData.Contains(AnswExpected))
                    {
                        ReceivedAnsw = true;
                        ReceivedLine = ReceivedCommandData;
                        ReceivedCommandData = "";
                    }
                }

                this.Invoke(new MethodInvoker(delegate
                {
                    AppendText(TextBox_System_Log, Color.Black, line + "\r\n");
                }
                ));
            }
        }
    }

I know that the problem is that buffer.ReadLine() treats remainder of the string which doesn't end with a CR or LF character as a seperate line but I don't know how to fix it.

I tried using port.ReadLine() in the past but it is way slower and causes problems for me when serial ports get disconnected etc.

Answer 1

You can try like below code:

    public void DataReceivedSerialPort(object sender, SerialDataReceivedEventArgs e)
{
    readExistingData = "";
    SerialPort sp = (SerialPort)sender;

    sp.ReadTimeout = 100;
    do
    {
        readExistingData = "";

        try
        {

            readExistingData = sp.ReadLine();
            if (readExistingData == "")
            {
                readExistingData = sp.ReadLine();
            }
            dataReadFromSerialPort += readExistingData;
        }
        catch
        {
            try
            {
                readExistingData = sp.ReadExisting();
                dataReadFromSerialPort += readExistingData + "\r\n";
            }
            catch { }

        }
        UI.insert_new_items_into_textBoxUARTLog(readExistingData);
    } while (readExistingData != "");
}

Answer 2

Your issue with \r\n and \n can be covered by using Environment.NewLine. I'm not sure what AppendText does, but if you're using it to store the values, then you're overdoing it. What you need is to store all data first in a StringBuilder then process them, OR process each data and store them in managed type such as Array, to define each line separately. Only use the string in the presentation layer (if you have some GUI that you want the user to see the results).

So, what I suggest is to store the lines in StringBuilder Something like this :

private readonly StringBuilder _strDataBuilder = new StringBuilder();

private void PrintSerialData()
{
    try
    {
        if (RandomReboot)
        {
            RebootWatch.Start();
        }
        
        if(COMport.IsOpen && COMport.BytesToRead != 0)
        {
            var data = COMport.ReadExisting();
            
            if(!string.IsNullOrEmpty(data)) {   
                _strDataBuilder.Append(data);
            }   
        }
        
    }
    catch (System.IO.IOException SerialException)
    {
        return;
    }   
}

private void SerialPrint()
{
    var data = _strDataBuilder.ToString();
    
    if(string.IsNullOrEmpty(data)) { return; }
    
    var lines = data.Split(Environment.NewLine);
    
    if(lines.Length == 0)  { return; }
    
    
    for(int x = 0; x < lines.Length; x++)
    {
        var line = lines[x];

        if (CheckForAnsw)
        {
            ReceivedCommandData = line;
            
            if (ReceivedCommandData.Contains(AnswExpected))
            {
                ReceivedAnsw = true;
                ReceivedLine = ReceivedCommandData;
                ReceivedCommandData = "";
            }
        }

        this.Invoke(new MethodInvoker(delegate
        {
            AppendText(TextBox_System_Log, Color.Black, line + Environment.NewLine);
        }
        ));     
    }   
}

Storing them first would make things more maintainability and fixability when you want to add more processing steps or reuse the results.

Although the SerialPrint() is unnessary if you just re-print the data in the GUI. As the data already separated in lines. So, if you do

TextBox_System_Log.Text = _strDataBuilder.ToString();

Directly, would list them in lines in the default color. However, if you intended to split them to process each line separately (to validate for instance), then it would be okay.

Answer 3

You can use AnonymousPipes to transport and buffer the incoming data and read them as lines to output them to somewhere.

Here is a little example which creates a server and client pipe stream, then writes data to the server in one task (with some newline in the data) and reads the data in a different task per line and outputs them to the console.

public class Program
{
    public static async Task Main()
    {
        (var writer, var reader) = CreatePipe();

        using (writer)
        using (reader)
        {
            var writerTask = Task.Run(async () =>
            {
                writer.AutoFlush = true;
                writer.Write("?");
                for (int i = 0; i < 100; i++)
                {
                    if (i % 10 == 9)
                    {
                        await writer.WriteAsync("!");
                        await writer.WriteAsync(Environment.NewLine);
                        await writer.WriteAsync("?");
                    }
                    else
                    {
                        await writer.WriteAsync((i % 10).ToString());
                    }
                    await Task.Delay(100);
                }
                writer.Close();
            });
            var readerTask = Task.Run(async () =>
            {
                while (!reader.EndOfStream)
                {
                    var line = await reader.ReadLineAsync();

                    Console.WriteLine(line);
                }
            });

            await Task.WhenAll(writerTask, readerTask);
        }
    }

    public static (StreamWriter, StreamReader) CreatePipe()
    {
        var server = new AnonymousPipeServerStream(PipeDirection.Out);
        var client = new AnonymousPipeClientStream(server.GetClientHandleAsString());

        return
            (
            new StreamWriter(server, Encoding.UTF8),
            new StreamReader(client, Encoding.UTF8)
        );
    }
}

Try to adapt this code to your use case and comment if there are difficulies.

Answer 4

I don't think there's an easy way to handle this with the StringReader. Instead, you can split the string yourself:

private static string _buffer = string.Empty;

private static void SerialPrint(string data)
{
    // Append the new data to the leftover of the previous operation
    data = _buffer + data;

    int index = data.IndexOf('\n');
    int start = 0;

    while (index != -1)
    {
        var command = data.Substring(start, index - start);
        ProcessCommand(command.TrimEnd('\r'));
        start = index + 1;
        index = data.IndexOf('\n', start);
    }

    // Store the leftover in the buffer
    if (!data.EndsWith("\n"))
    {
        _buffer = data.Substring(start);
    }
    else
    {
        _buffer = string.Empty;
    }
}

private static void ProcessCommand(string command)
{
    Console.WriteLine(command);
}