Processing text coming off a serial line in C#

Question

I've got an embedded device running Linux that I can communicate with via RS232. I'm attempting to programmatically control it from within an existing .NET application, so I've been fooling around with the System.IO.Ports.SerialPort class. I'm reading data off the port right now by using the SerialPortDataRecieved event handler and checking for the amount of bytes to read, then passing in a byte[] of that size, THEN shoving that byte[] into a Queue<byte[]> for further processing, like this:

public class SerialTalk{
SerialPort s;    
Queue<byte[]> RecQueue=new Queue<byte[]>;
//...bunch of irrelevant stuff here (ctors etc)
   private void DataReceivedHandler(Object sender, SerialDataReceivedEventArgs e)
   {
        byte[] recv;
        lock (s_lock)
        {
            int bytes = s.BytesToRead;
            recv = new byte[bytes];
            s.Read(recv, 0, bytes);
        }

        Console.Write(recv);
        lock (RecQueue)
        {
            RecQueue.Enqueue(recv);
        }
   }
}

Another thread then reads from that queue to perform processing, printing to the terminal, etc. This has been working great for printing, but I want to be able to regex the data I'm retrieving. The problem is, the amount of bytes returned by the SerialPort is unpredictable and completely unaware of what information it is sending, so the chunks of data I'm getting are, generally, fragments of lines.

What I'm looking for is a simple, performant, multithreaded way to go from line fragment byte[] to strings that are organized neatly according to lines. One idea I had was to have another thread looping infinitely that would Dequeue byte[]s from the queue, turn them into strings, append them to a StringBuilder, then split the string returned from StringBuilder.ToString() along the newlines, the take the LAST string and use it to construct a new StringBuilder for the next time around, then enqueue all the other strings into a Queue<string> -- this way, fragments would get put back together... but this won't work in the most important case of all, which is when I'm waiting to find the shell prompt, since that string won't ever get put into the second queue.

Ideas?

Answer 1

I would Just buffer characters in DataReceivedHandler until you detected a newline and then Queue the chars before the newline (\r\n) as a string and keep the remaining chars in the buffer until the next newline. Usually for \r\n situations I always scrub the \r's and only interpret the \n's.

Here's a quick mod to your source for illustration.

public class SerialTalk{
SerialPort s;    
Queue<string> RecQueue=new Queue<string>();
//...bunch of irrelevant stuff here (ctors etc)
string innerBuffer = ""; //a buffer we keep in between DataReceiveHandlers for the portion of text not newlined yet
private void DataReceivedHandler(Object sender, object e)
{
    byte[] recv;

    lock (s_lock)
    {
        int bytes = s.BytesToRead;
        recv = new byte[bytes];
        s.Read(recv, 0, bytes);
    }
    innerBuffer += System.Text.Encoding.GetEncoding("utf-8").GetString(recv).Replace("\r\n","\n"); //could be enhanced using stringbuilder.
    string[] lines = innerBuffer.Split('\n'); 
    innerBuffer = lines[lines.Length-1];  //always keep the portion after the last newline in the buffer
    Console.Write(recv);
    lock (RecQueue)
    {
        for(int x=0;x<innerBuffer.Length-2;x++) //-2 here instead of -1 as you don't want to send the remainder of the inner buffer until the next newline
            RecQueue.Enqueue(lines[x]);
    }
}
}

*should be noted that converting to utf-8 from your byte array may cause additional issues. If you still want to send it as a byte[] into the Queue, you can still keep a buffer locally like the innerBuffer string does, but \r\n delineation marks won't be quite as trivial to find as Split, although still not a big deal.

Answer 2

You can do a small state machine. For each byte received, if it's not a newline, add it to the 'current string' data.

If it is a newline, fire the 'new-data'; event.

If you detect that the 'current string' is equal to the expected 'shell prompt', fire the 'new-data' event.

Perform that processing for each byte received in turn. That doesn't stop you reading all available bytes from the port at once, of course.