Asynchronous IO on serial port

Question

I need your advices in designing a fully asynchronous IO on a half-duplex serial port. Currently I have a reader thread and many writer threads controlled by semaphores and a mutex. Now I want to simplify synchronization by eliminating threads. The main problem is that serial port IO has a strange behavior.

All I need is to make sure that read and write system calls block only calling thread until IO operation is actually done. I'm assuming read is a blocking system call by default. Though I'm getting -1 as return of read. There is a strange EBUSY error which I have no description for. Current code:

bool SerialManager::initialize(const PortType& portType, const size_t& number)
{
// Open Serial port (/dev/ttyS2 in this case)
fd = open(portName.str().c_str(), O_RDWR ); //O_NOCTTY
if (fd < 0) // if open is not successful
{
    cerr << ERROR << "Unable to open `" << portName << "'." << endl;
    return false;
}
else
{
    cout << INFO << "Port " << portName.str() << " successfully opened."
            << endl;
    cout << INFO << "Configuring port..." << endl;
    fcntl(fd, F_SETFL,~O_NONBLOCK);
    struct termios port_settings; // structure to store the port settings in
    cfsetispeed(&port_settings, B38400); // set baud rate
    cfsetospeed(&port_settings, B38400); // set baud rate
    port_settings.c_cflag |= CLOCAL | CREAD;
    port_settings.c_cflag &= ~CRTSCTS; // disable H/W flow control
    port_settings.c_lflag &= ~( ISIG | // disable SIGxxxx signals
            IEXTEN | // disable extended functions
            ECHO | ECHOE); // disable all auto-echo functions
    port_settings.c_lflag &= ~ICANON ; // raw mode
    port_settings.c_oflag &= ~OPOST; // raw output
    port_settings.c_iflag &= ~(IXON | IXOFF | IXANY); // disable S/W flow control;
    port_settings.c_cc[VTIME] = 20; // wait 0.1 second to get data
    port_settings.c_cc[VMIN] = 0;

    port_settings.c_cflag = (port_settings.c_cflag &= ~CSIZE) | CS8; // set data byte size
    port_settings.c_cflag &= ~CSTOPB; // set stop bit 1
    port_settings.c_cflag &= ~PARENB; // set no parity
    port_settings.c_iflag |= IGNPAR; // ignore parity
    port_settings.c_iflag &= ~(INPCK | ISTRIP | PARMRK);

    // Set
    if (tcsetattr(fd, TCSANOW, &port_settings) < 0)
    {
        cerr << ERROR << "Unable to configure serial port." << endl;
        return false;
    }
    else
    {
        cout << INFO << "Port `" << portName.str()
                << "' configuration was successful." << endl;
            return true;
    }
}
}

To write data:

int SerialManager::asyncWriteData(const byte* data, const size_t& size)
{
    int writeSize = write(fd, data, size);
    return writeSize;
}

For read:

void SerialManager::asyncRead(byte* buffer, const size_t& size, bool& ok)
{
    byte temp[256];
    ssize_t packetSize = read(fd, temp, 256);
    if (packetSize > 0)
    {
        for (size_t i = 0; i < size; ++i)
            buffer[i] = temp[i];
        ok = true;
    }
    cout << errno << endl;
    perror("Error occured: "); // <=== Here I'm getting EBUSY (code 16)
    ok = false;
}

Using SerialManager class outside:

....
word checksum = this->id + 0x2C;
checksum = ~checksum;
// Send read command
byte command[] =
{ 0xff, // heading
        0xff, // ~
        this->id, // id of actuator
        0x04, // length
        0x02, // instruction: read
        0x24, // start address: present position
        0x02, // data length
        static_cast<byte>(checksum) //checksum
        };
SerialManager::lockPort(); // lock a mutex to avoid collitions
int numbytes = SerialManager::asyncWriteData(command, 8);
if (numbytes < 0)
{
    cerr << ERROR << "Could not write to serial port." << endl;
    return 0;
}
cout << INFO << numbytes << " bytes has been written." << endl;
for (size_t i = 0; i < 8; ++i)
{
    cout << hex << setfill('0') << setw(2) << (int) command[i] << ' ';
}
cout << endl;

byte* data = new byte[8];
bool ok;
// Here I need to make sure data write is completed before start reading
SerialManager::asyncRead(data, 8, ok);
if (ok)
{
    word position = data[5] + (static_cast<word>(data[6]) << 8);
    return position;
}
else
{
    cerr << ERROR << "Unable to read data from serial port..." << endl;
    return -1;
}
SerialManager::unlockPort(); // Unlock previously locked mutex
....

Update:

I removed reader thread which makes no sense. Because we have a half-duplex line with no control over transmission. There are two problems with synchronous IO:

1. Sending a very long data from controller to actuators, first actuator responds unresponsively while data is on port:

2. An actuators may respond while another one's data is not transmitted completely

Also problem with EBUSY is solved by adding a fsync after write. Which is somehow what I need. (A blocking write):

int SerialManager::asyncWriteData(const byte* data, const size_t& size)
{
    ssize_t packetSize = write(fd, data, size);
    if (packetSize > 0)
    {
        fsync(fd);
    }
    return packetSize;
} 

from man fsync:

fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) where that file resides. The call blocks until the device reports that the transfer has completed. It also flushes metadata information associated with the file

Answer 1

I doubt this answers your question, but I think it will help.

In void SerialManager::asyncRead(byte*, const size_t&, bool&), there are two bugs:

1. You're using the value of errno (and printing messages based on it using perror) when read returns success, not only when it errors out (return value < 0). On success, errno is not specified to have any particular value. So you're effectively using garbage data; the EBUSY you're getting probably doesn't mean anything whatsoever. errno is only defined to have a particular value when a particular error has occurred. See 'man errno' on a Linux system, which on my Debian system says in relevant part: "Its value is significant only when the return value of the call indi‐cated an error (i.e., -1 from most system calls; -1 or NULL from most library functions); a function that succeeds is allowed to change errno.". Note the "is allowed to", rather than "must".

2. You read into a temp buffer, then copy it byte-by-byte into the destination buffer which the caller passed. This is a pointless waste of CPU cycles and memory bandwidth. You could just read straight into the caller's buffer. Even if you must (for some strange reason) read into a temp buffer then copy it elsewhere, you should copy it elsewhere using memcpy(3) or perhaps memmove(3), not using your own byte-by-byte loop.

Regarding the design, I don't understand the point of having multiple readers or writers when there is only one serial port. Are you trying to achieve parallelism? I don't see how you can, with only one serial port. If you want asynchronicity but not parallelism, you should investigate fcntl(fd, F_SETFL, existing_flags & O_ASYNC) (although signals are rather old and ugly), or perhaps just set the fd non-blocking, then call select() in the inner event loop of your application and handle any I/O (if any) that's available.

Hope this helps!