CODEWITHSUNDEEP
linuxembeddedtcp
Michael Kohne
Michael Kohne

Reputation: 12044

I need a TCP option (ioctl) to send data immediately

I've got an unusual situation: I'm using a Linux system in an embedded situation (Intel box, currently using a 2.6.20 kernel.) which has to communicate with an embedded system that has a partially broken TCP implementation. As near as I can tell right now they expect each message from us to come in a separate Ethernet frame! They seem to have problems when messages are split across Ethernet frames.

We are on the local network with the device, and there are no routers between us (just a switch).

We are, of course, trying to force them to fix their system, but that may not end up being feasible.

I've already set TCP_NODELAY on my sockets (I connect to them), but that only helps if I don't try to send more than one message at a time. If I have several outgoing messages in a row, those messages tend to end up in one or two Ethernet frames, which causes trouble on the other system.

I can generally avoid the problem by using a timer to avoid sending messages too close together, but that obviously limits our throughput. Further, if I turn the time down too low, I risk network congestion holding up packet transmits and ending up allowing more than one of my messages into the same packet.

Is there any way that I can tell whether the driver has data queued or not? Is there some way I can force the driver to send independent write calls in independent transport layer packets? I've had a look through the socket(7) and tcp(7) man pages and I didn't find anything. It may just be that I don't know what I'm looking for.

Obviously, UDP would be one way out, but again, I don't think we can make the other end change anything much at this point.

Any help greatly appreciated.

Upvotes: 7

Views: 8638

Answers (6)

Tommy
Tommy

Reputation: 1038

You cannot work around a problem unless you're sure what the problem is.

If they've done the newbie mistake of assuming that recv() receives exactly one message then I don't see a way to solve it completely. Sending only one message per Ethernet frame is one thing, but if multiple Ethernet frames arrive before the receiver calls recv() it will still get multiple messages in one call.

Network congestion makes it practically impossible to prevent this (while maintaining decent throughput) even if they can tell you how often they call recv().

Upvotes: 2

Steve Baker
Steve Baker

Reputation: 4373

Maybe you could try putting the tcp stack into low-latency mode:

echo 1 > /proc/sys/net/ipv4/tcp_low_latency

That should favor emitting packets as quickly as possible over combining data. Read the man on tcp(7) for more information.

Upvotes: -1

Alexander
Alexander

Reputation: 9370

In the worst case scenario you could go one level lower (raw sockets), where you have better control over the packets sent, but then you'd have to deal with all the nitty-gritty of TCP.

Upvotes: 0

Adam Liss
Adam Liss

Reputation: 48280

Have you tried opening a new socket for each message and closing it immediately? The overhead may be nauseating,but this should delimit your messages.

Upvotes: 0

ADEpt
ADEpt

Reputation: 5542

Maybe, set TCP_NODELAY and set your MTU low enough so that there would be at most 1 message per frame? Oh, and add "dont-fragment" flag on outgoing packets

Upvotes: 1

Martin v. Löwis
Martin v. Löwis

Reputation: 127447

IIUC, setting the TCP_NODELAY option should flush all packets (i.e. tcp.c implements setting of NODELAY with a call to tcp_push_pending_frames). So if you set the socket option after each send call, you should get what you want.

Upvotes: 11

Related Questions