CODEWITHSUNDEEP
c++timestampserial-portboost-asio
OcularProgrammer
OcularProgrammer

Reputation: 482

Timestamping incoming serial data under WinXP 32 using Boost ASIO and Boost posix time?

I'm trying to read in and timestamp data from a serial device under WinXP 32. I'm using boost::asio for my serial recieve, but have been running into some limitations. If I try to read and timestamp each 2-byte range measurement by telling asynch_read_some to use a 2-byte buffer, I get massive latency on my range measurements.

To get around this my plan has been to use large (5 kb) buffers, timestamp the time when I receive each buffer and dump the buffer, a timestamp, and some other stuff (size, header, footer, etc) into a file. I need these timestamps, as I have to synchronise the sensor data with data from a different device on the same PC. The logging process for that device is also applying timestamps in much the same way, and I am going to hook it all up offline after I have taken my logs.

This function is called directly from my boost::asio serial RX callback as soon as data is recieved, and passed the pointer to the buffer and the number of bytes recieved.

void AR2500CallbackFunction(char * RXBuff, unsigned int bytesRcvd)
{
  boost::posix_time::ptime nowPTime( boost::posix_time::microsec_clock::local_time() );
  uint64_t pktTimestamp( nowPTime.time_of_day().total_microseconds() );

  // Log Packet Header - PacketStartSentinel                                                    
  m_logFileStream.write((char *)PacketStartSentinel, sizeof(PacketStartSentinel));
  // Log Packet ID - m_packetCounter
  m_logFileStream.write((char *)&m_packetCounter, sizeof(m_packetCounter));
  // Add number of bytes recieved - bytesRcvd
  m_logFileStream.write((char *)&bytesRcvd, sizeof(bytesRcvd));
  // Log Data - RXBuff
  m_logFileStream.write(RXBuff, bytesRcvd);
  // Add Timestamp - pktTimestamp
  m_logFileStream.write((char *)&pktTimestamp, sizeof(pktTimestamp));
  // Add End Sentinel - PacketEndSentinel
  m_logFileStream.write((char *)PacketEndSentinel, sizeof(PacketEndSentinel));
  m_packetCounter++;
}

Since my data is coming in as a constant stream over the serial port, I then make what seems to be a reasonable assumption. I assume that the timestamp I record with each packet is a repeatable (but not necessarily accurate, there will be some approximately constant lag before the callback actually executes) measurement of the time at which I received the last sample in the packet. In my post processing stage, I then take the timestamp from the the previous packet, the timestamp of the current packet, and linearly interpolate to estimate the timestamp of each of the samples within the current packet.

What I am finding after post-processing is that, even for extremely low data-rates where there is only 1 or 2 range readings per callback, I am getting extremely unstable timestamping. Some packets are stamped as having the same time, some consecutive packets are being stamped as having impossibly large gaps. When I set the sensor sampling rate as low as 10 Hz, I still get some timestamp differences between samples of 125 milliseconds, and some as low as 80 milliseconds. If I turn the sampling rate up (the sensor can return 2-byte range readings at up to 30 KHz over a 921600 baud RS232 link) I find that the instability becomes even worse. I know by using an oscilloscope on the serial lines that the sensor is actually spitting the data out at the commanded rate, so the flaw must be somewhere in the software-side.

Is there something that I am doing wrong, a serious flaw in one of my assumptions, or am I simply going to have to do this all under linux instead to get accurate timestamps? I'd really rather avoid this last option, since although my serial code is cross-platform, my other device code would be hard to make portable.

Upvotes: 0

Views: 698

Answers (1)

Martin James
Martin James

Reputation: 24907

Timestamp the data external to the PC or use a real-time OS. Desktop OS that are really good at running Office and Firefox are spectacularly hopeless at real-time data acquisition. The drivers are optimized to signal the receipt of huge data buffers so as to maximize throughput of complex web pages and streamed media. Prompt signaling of the receipt of a couple of bytes, (or even 5K), is a bit below them - the drivers can't be bothered and tend to lie around for ages in case more data comes in, then they can signal it all at once :((

All my embedded controllers that return data that needs to be time-stamped return the stamp in the data and do not rely on some megalithic desktop OS to wake up a thread in any sort of prompt manner.

That said, you might be able to tweak the serial driver COMMTIMEOUTS to return data more quickly. Whether this will help much is another matter because you have other devices that may be subject to the same issues.

Also, what is 'm_logFileStream'? Is this another async or otherwise queued disk write? If not, it should be.

If you want to try to apply timestamps to data in the PC in the way you seem to want, then ideally all your data should flow through the same thread that adds the timestamps and does the logging. Writing multiple logs and filtering them afterwards to sort them into timestamp order is just going to add yet more jitter. That sort of thing is what Unix developers do because their computers can only execute bash scripts :))

Upvotes: 0

Related Questions