How to (almost) prevent FT232R (uart) receive data loss?

Question

I need to transfer data from a bare metal microcontroller system to a linux PC with 2 MBaud. The linux PC is currently running a 32 bit Kubuntu 14.04.

To archive this, I'd tried to use a FT232R based USB-UART adapter, but I sometimes observed lost data.

As long as the linux PC is mainly idle, it seems to work most time; however, I see rare data loss.

But when I force cpu load (e.g. rebuild my project), the data loss increases significantly.

After some research I read here, that the FT232R consist of a receive buffer with a capacity of only 384 Byte. This means, that the FT232R has to be read out (USB-polled) after at least every 1,9 ms. Well, FTDI recommends to use flow control, but because of the used microcontroller system, I'm fixed to cannot use any flow control.

I can live with the fact, that there is no absolutely guarantee for having no data loss. But the observed amount of data loss is quiet too heavy for my needs.

So I tried to find a way to increase the priority of the "FT232 driver" on my linux, but cannot find how to do this. It's not described in the AN220 FTDI Drivers Installation Guide for Linux and the document AN107 FTDI Advanced Driver Options has a capter about "Changing the Driver Priority" but only for windows.

So, does anybody know how to increase the FT232R driver priority in linux?

Any other ideas to solve this problem?

BTW: As I read the FT232H datasheet, it seems that this comes with 1 KiB RX buffer. I'd order one just now and check out its behaviour. Edit: No significant improvement.

Answer 1

If you want reliable data transfer, there is absolutely no way to use any USB-to-serial bridge correctly without hardware flow control, and without dedicating at least all remaining RAM in your microcontroller as the serial buffer (or at least until you can store ~1s worth of data).

I've been using FTDI devices since FT232AM was a hot new thing, and here's how I implement them:

1. (At least) four lines go between the bridge and the MCU: RXD, TXD, RTS#, CTS#.

2. Flow control is enabled on the PC side of things.

3. Flow control is enabled on the MCU side of things.

4. MCU code is only sending communications when it can fit a complete reply packet into the buffer. Otherwise, it lets the PC side of it time out and retry the request. For requests that stream data back, the entire frame is dropped if it can't fit in the transmit buffer at the time the frame is ready.

5. If you wish the PC to be reliably notified of new data, say every number of complete samples/frames, you must use event characters to flush the FTDI buffers to the hist, and encode your data. HDLC works great for that purpose and is documented in free standards (RFCs and ITU X and Q series - all free!).

6. The VCP driver, or the D2XX port bring-up is set up to have transfer sizes and latencies set for the needs of the application.

7. The communication protocol is framed, with CRCs. I usually use a cut-down version if X.25/Q.921/HDLC, limited to SNRM(E) mode for simple "dumb" command-and-respond devices, and SABM(E) for devices that stream data.

The size of FTDI buffers is immaterial, your MCU should have at least an order of magnitude more storage available to buffer things.

If you're running hard real-time code, such as signal processing, make sure that you account for the overhead of lots of transmit interrupts running "back-to-back". Once the FTDI device purges its buffers after a USB transfer, and indicates that it's ready to receive more data from your MCU, your code can potentially transmit a full FTDI buffer's worth of data at once.

If you're close to running out of cycles in your realtime code, you can use a timer as a source of transmit interrupts instead of the UART interrupt. You can then set the timer rate much lower than the UART speed. This allows you to pace the transmission slower without lowering the baudrate. If you're running in setup/preoperational mode or with lower real-time task load, you can then trivially raise the transmit rate without changing the baudrate. You can use a similar trick to pace the receives by flipping the RTS# output on the MCU under timer control. Of course this isn't a problem is you use DMA or a sufficiently fast MCU.

If you're out of timers, note that many other peripherals can also be repurposed as a source of timer interrupts.

This advice applies no matter what is the USB host.

^{Sidebar: Admittedly, Linux USB serial driver "architecture" is in the state of suspended animation as far as I can tell, so getting sensible results there may require a lot of work. It's not a matter of a simple kernel thread priority change, I'm afraid. Part of the reason is that funding for a lot of Linux work focuses on server/enterprise applications, and there the USB performance is a matter of secondary interest at best. It works well enough for USB storage, but USB serial is a mess nobody really cares enough to overhaul, and overhaul it needs. Just look at the amount of copy-pasta in that department...}