CODEWITHSUNDEEP
tcpgo
Charlie Parker
Charlie Parker

Reputation: 5221

How do you read without specifying the length of a byte slice beforehand, with the net.TCPConn in golang?

I was trying to read some messages from a tcp connection with a redis client (a terminal just running redis-cli). However, the Read command for the net package requires me to give in a slice as an argument. Whenever I give a slice with no length, the connection crashes and the go program halts. I am not sure what length my byte messages need going to be before hand. So unless I specify some slice that is ridiculously large, this connection will always close, though this seems wasteful. I was wondering, is it possible to keep a connection without having to know the length of the message before hand? I would love a solution to my specific problem, but I feel that this question is more general. Why do I need to know the length before hand? Can't the library just give me a slice of the correct size?

Or what other solution do people suggest?

Upvotes: 0

Views: 1867

Answers (2)

prr
prr

Reputation: 68

A bit late but...

  1. One of the questions was how to determine the message size. The answer given by JimB was that TCP is a streaming protocol, so there is no real end.
  2. I believe this answer is incorrect. TCP divides up a bitstream into sequential packets. Each packet has an IP header and a TCP header See Wikipedia and here. The IP header of each packet contains a field for the length of that packet. You would have to do some math to subtract out the TCP header length to arrive at the actual data length. In addition, the maximum length of a message can be specified in the TCP header.
  3. Thus you can provide a buffer of sufficient length for your read operation. However, you have to read the packet header information first. You probably should not accept a TCP connection if the max message size is longer than you are willing to accept.
  4. Normally the sender would terminate the connection with a fin packet (see 1) not an EOF character.
  5. EOF in the read operation will most likely indicate that a package was not fully transmitted within the allotted time.

Upvotes: 1

JimB
JimB

Reputation: 109339

Not knowing the message size is precisely the reason you must specify the Read size (this goes for any networking library, not just Go). TCP is a stream protocol. As far as the TCP protocol is concerned, the message continues until the connection is closed.

If you know you're going to read until EOF, use ioutil.ReadAll

Calling Read isn't guaranteed to get you everything you're expecting. It may return less, it may return more, depending on how much data you've received. Libraries that do IO typically read and write though a "buffer"; you would have your "read buffer", which is a pre-allocated slice of bytes (up to 32k is common), and you re-use that slice each time you want to read from the network. This is why IO functions return number of bytes, so you know how much of the buffer was filled by the last operation. If the buffer was filled, or you're still expecting more data, you just call Read again.

Upvotes: 7

Related Questions