what is the right network protocol to use to transfer an integer number sporadically from computer A to B with lowest possible latency?

Question

I'm not a network expert so please bear with me. I need to establish a very simple one-directional data connection between two computers, which will be used ~ every 100ms to transfer an integer number (int8 or int16), from computer A to computer B. The connection and data transfer will always be in the direction A -> B. Since I want to react as quickly as possible to packages coming in @ B, I'd like to know which protocol I should use in order to minimize network latencies. That is, I want the transfer to be as 'realtime' as possible, ideally with latencies being < 10ms (and no packages should get lost if possible).

So here are my specific questions:

1. what communication protocols are suited for this job?
2. is this latency that I'm trying to achieve realistic?
3. what is the minimum latency that I could theoretically achieve?
4. will the latency depend drastically on the switch used? Or will any cheap consumer switch do fine?
5. in case I need to suddenly send more data, what are the optimal package sizes in order to minimize overhead?
6. what's the best / correct way to actually measure latency?
7. how much will the latency depend on the programming language used? (I'm considering java, c, python)
8. what would the pros and cons be for using TCP?
9. what would the pros and cons be for using UDP?
10. what would the pros and cons be for using websockets?

Computers A and B are connected to the same switch and are physically very close by and in order to reduce unnecessary 'network noise', the computers could be separated from the rest of the network.

Sample code is greatly appreciated :)

Answer 1

Answers:

1. what communication protocols are suited for this job?

   It kind of depends on your application's tolerance of packet loss vs latency jitter (latency variability). In your local single switch configuration, you probably won't run into either of those issues no matter what transport you use, but if your network becomes noisy or you might ever decide to extend this out over a larger network then you will need to decide which is more important. If avoiding packet loss is your highest priority then you should use a session based transport (TCP, WebSockets). If avoiding latency jitter is paramount then you probably want to use a UDP transport. For example, most online FPS games use UDP because occasional packet loss or re-ordering is acceptable, but latency jitter is not.

2. is this latency that I'm trying to achieve realistic?

   Yes, see next answer.

3. what is the minimum latency that I could theoretically achieve?

   On a local single switch network it definitely possible to get average latencies under 10 milliseconds. If you use Gigabit ethernet end-to-end you could likely achieve an average one way latency of under 0.1 milliseconds (100 microseconds).

4. will the latency depend drastically on the switch used? Or will any cheap consumer switch do fine?

   The switch will definitely affect your latency. However, even a cheap 100 Mb/s consumer switch should get you under 10ms (probably even under 1ms).

5. in case I need to suddenly send more data, what are the optimal package sizes in order to minimize overhead?

   As a general rule, the lowest overhead will be achieved by using the maximum payload that will not be fragmented. But you have to consider fragmentation end-to-end. For example, do your network cards and switch all support jumbo frames (larger than 1500 bytes)? Do they all have the same MTU setting? Does your transport library have an arbitrary buffering size?

6. what's the best / correct way to actually measure latency?

   I would suggest creating a latency test application that incorporates the actual hardware and all the whole software stack that you intend to use and build a latency test application to test actual practical latency (not just theoretical). You will want to test round trip latency and then divide by two (otherwise you will spend all your time on high accuracy time synchronization). Also, you will want to be able to adjust payload size easily especially if you are wanting to answer the previous question about the most efficient package/message size. If you are considering UDP then I also recommend adding sequence numbers to your payload and checking them to make sure you aren't experiencing any unexpected packet loss or re-ordering (re-ordering shouldn't happen with a one switch network).

   If you are going to use large payloads/packages, then I also highly recommend using wireshark to see what is actually happening on the wire. You should be able to easily identify any fragmentation that is happening (and it's a great way to gain a new understanding of how networking protocols work).

7. how much will the latency depend on the programming language used? (I'm considering java, c, python)

   Choice of language can certainly make a non-trivial difference. However, in your single switch local network case, I suspect that python will be sufficient since should be well below the margin. In Java, and Python the socket code is implemented in C/C++ so as long as there isn't much pre or post processing of the data, the additional overhead shouldn't be too noticeable. In my opinion, the bigger risk of using python or Java is if your application is sensitive to latency jitter. Java and python are both garbage collected languages and depending on the nature of your application this can introduce unexpected jitter in the effective latency (i.e. you make get occasional pauses while the language runtime garbage collects).

   I would personally start with python because development is so rapid (but start with whichever is most efficient for you) and then switch to C if performance turns out to be an issue. Also, if there is any chance that your project will be incorporated into a larger ecosystem, then you should probably choose the preferred language within that ecosystem.

8. what would the pros and cons be for using TCP?

   Pros: Guaranteed in-order delivery of all data.

   Cons: Requires more initial setup. With a complex or noisy network environment may have latency jitter due to retries and re-ordering.

9. what would the pros and cons be for using UDP?

   Pros: Low latency.

   Cons: Only best-effort. Packets may be dropped and/or re-ordered. Can be more complicated to cross firewalls.

10. what would the pros and cons be for using websockets?

    Pros: Allows a browser to be one of your end-points, otherwise same Pros as TCP (it is a transport layered on TCP). It is a message based transport so no re-assembly of data is necessary (e.g. you receive whole messages as they were sent).

    Cons: Has HTTP like handshake and some minimal framing for each message.

Here is a simple round-trip echo client and server in python:

Server:

# Echo server program
import socket, sys

while True:
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(('', int(sys.argv[1])))
    s.listen(1)
    conn, addr = s.accept()
    print 'Connected by', addr
    while True:
        data = conn.recv(1024)
        if not data: break
        conn.sendall(data)
    conn.close()

Client:

# Echo client program
import socket, sys, time

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((sys.argv[1], int(sys.argv[2])))
i = 0
while True:
    i += 1
    start = time.time()
    s.sendall('Test payload #%d' % i)
    data = s.recv(1024)
    end = time.time()
    print "%f-%f" % (end, start)
    print "Received '%s' back in %f milliseconds" % (data,
            (end-start)*1000.0)
    time.sleep(0.5)

On your server host, run:

python ./server.py 8123

On your client host, run:

python ./client.py SERVER_HOST 8123

Answer 2

I think the solution you're looking for is TCP sockets in a client server architecture. As you didn't state the choice of programming language, I'd first try a proof-of-concept in Python and then (if it's too slow) switch to either plain C or C++ (using one of the libs mentioned here).

Also, have a look here for latency and sockets.

Answer 3

If I understand correctly, the requirements should be very easy to meet. In fact, you should probably be able to get < 1ms latency. A simple/crude test would be to use ping between the two computers to get a rough idea of the time.

For the implementation, I would use TCP/IP. It makes the reliability simpler to achieve (simpler than with UDP) and is relatively easy to implement. You will probably want to disable the Nagle algorithm (setsockopt( ... TCP_NODELAY ...)) to eliminate delays that would normally happen with such small packets.

Answer 4

Giving those very optimal network conditions, I'd say UDP is perfect for that task (data loss should be so rare that TCP is an overkill).

Answer 5

You can use TCP for this. Just make sure to have B send an application-level acknowledgement back to A after it receives each integer. That will allow the ACK to piggy-back on the data, ensuring the next transfer is not delayed. Less than 1ms latency should be possible.