I'm programming a server that accepts an incoming connection from a client and then reads from it (via net.Conn.Read()). I'm going to be reading the message into a []byte slice obviously and then processing it in an unrelated way, but the question is - how do I find out the length of this message first to create a slice of according length?
Asked
Active
Viewed 211 times
3 Answers
2
It is entirely dependent on the design of the protocol you are attempting to read from the connection.
If you are designing your own protocol you will need to design some way for your reader to determine when to stop reading or predeclare the length of the message.
For binary protocols, you will often find some sort of fixed size header that will contain a length value (for example, a big-endian int64) at some known/discoverable header offset. You can then parse the value at the length offset and use that value to read the correct amount of data once you reach the offset beginning the variable length data. Some examples of binary protocols include DNS and HTTP/2.
For text protocols, when to stop reading will be encoded in the parsing rules. Some examples of text protocols include HTTP/1.x and SMTP. An HTTP/1.1 request for example, declares the protocol to look something like:
METHOD /path HTTP/1.1\r\n
Header-1: value\r\n
Header-2: value\r\n
Content-Length: 20\r\n
\r\n
This is the content.
The first line (where line is denoted as ending with \r\n) must include the HTTP method, followed by the path (could be absolute or relative), followed by the version.
Subsequent lines are defined to be headers, made up of a key and a value.
The key includes any text from the beginning of the line up to, but not including, the colon. After the colon comes a variable number of insignificant space characters followed by the value.
One of these headers is special and denotes the length of the upcoming body: Content-Length. The value for this header contains the number of bytes to read as the body. For our simple case (ignoring trailers, chunked encoding, etc.), we will assume that the end of the body denotes the end of the request and another request may immediately follow.
After the last header comes a blank line denoting the end of the header block and the beginning of the body (\r\n\r\n).
Once you are finished reading all the headers you would then take the value from the Content-Length header you parsed and read the next number of bytes corresponding to its value.
For more information check out:
Adam Tanner
- 897
- 6
- 9
-
2Related question and answer [here](http://stackoverflow.com/questions/26999615/go-tcp-read-is-non-blocking), touching on a few relevant APIs in the Go stdlib (`net/textproto`, `io.ReadAtLeast` and `io.ReadFull`). – twotwotwo Jul 04 '16 at 07:09
-
My server is supposed to read basic text messages from a connection (it's more of a learning task than a real one right now), so I'm not really using existing protocols. – Arthmost Jul 04 '16 at 18:00
-
1@Arthmost: you need a "protocol" of _some_ sort. Whether it's predetermined message size, length prefixed strings, or `:-)` terminated, you need to way to know when you've read a complete message from the stream. – JimB Jul 05 '16 at 13:50
0
In the end what I did was create a 1024 byte slice, then read the message from the connection, then shorten the slice to the number of integers read.
Arthmost
- 137
- 7
0
The solution selected as correct is not good. What happens if the messages is more than 1024 Bytes? You need to have a protocol of the form TLV (Type Length Value) or just LV if you don't have different types of messages. For example, Type can be 2 Bytes and Length can be 2 Bytes. Then you first always read 4 Bytes, then based on the length indicated in Bytes 2 and 3, you will know how many Bytes come later and then you read the rest. There is something else you need to take into account: TCP is stream oriented so in order to read the complete TCP message you might need to read many times. Read this (it is for Java but useful for any language): How to read all of Inputstream in Server Socket JAVA
