Reading more than one "message" from recv()

Question

It is entirely possible that more than one separate "messages" (e.g. 2 send()s) could be read into a buffer using the recv() call.

In such a case, how would you put the second message back into the recv() buffer, once you've realised you have more data in your buffer than you need?

For example,

All messages are prepended with a byte dictating their length. I need to keep receiving until the correct number of bytes has been read into a buffer, but not continue beyond that point.

One idea is to do one recv() to establish message length, and then create a buffer with that size. I don't know what would happen though to data which doesn't fit into the buffer.

Answer 1

how would you put the second message back into the recv() buffer, once you've realised you have more data in your buffer than you need?

Just do not put the second message out of the recv() buffer, I see two ways for that :

1) do first

ssize_t size = recv(sockfd, buf, len, MSG_PEEK | MSG_TRUNC);

• MSG_TRUNC (AF_PACKET only) gives you the real size of the available data not the possibly length truncated to len
• with MSG_PEEK the received data is not removed from the queue.

That allows you to analyze the peek data and

• if it is a sub part of the first message but not its end you read (not peek) it recv(sockfd, buf, size); then you redo the previous recv etc
• if you have the (end of the) first message and may be a part of the second message you know the subSize you need to read and do recv(sockfd, buf, subSize);, and your second message is still available for the next revc

Of course each time you read a sub part of the first message the pointer buf progress to not rewrite the already read portion.

Use malloc then realloc to increase the size of the buffer receiving the first message

2) a very common way it to send the size of the message before the message itself, that allow the receiver to first read the size then to read the data in a loop until all the message is read. To be compatible with little/big endian if the message is greater than 255 bytes use htons/htonl/ntohs/ntohl for the size

I don't know what would happen though to data which doesn't fit into the buffer.

If you speak about to write out of the buffer nobody knows because the behavior is undefined, if you have chance you have a segmentation fault which visible contrarily to a dramatic memory corruption whose effects can be visible very late. But as you can see in the two solutions above this case fortunately does not happen

---

Example for the first case, using TCP/IP (no MSG_TRUNC then), the space indicates the end of each buffer (but I do not read character by character, to be compatible with a more complicated determination of the buffer end).

The server get in argument the strings to send, each argument is sent in one send whatever it contains or not a spaces, the last character of the last argument must be a space.

The client get one argument being the size to (try to) read each time, it prints each 'peek' buffer (for debug) and each buffer.

server.c

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <errno.h>

int main(int argc, char ** argv)
{
  errno = 0;

  int ssock = socket(AF_INET, SOCK_STREAM, 0);

  if (ssock == -1) {
    perror("socket()");
    return -1;
  }

  int reuse = 1;
  if (setsockopt(ssock, SOL_SOCKET, SO_REUSEADDR, (const char*)&reuse, sizeof(reuse)) == -1) {
    perror("setsockopt() SO_REUSEADDR)");
    return -1;
  }

#ifdef SO_REUSEPORT
  if (setsockopt(ssock, SOL_SOCKET, SO_REUSEPORT, (const char*)&reuse, sizeof(reuse)) == -1) {
    perror("setsockopt() SO_REUSEPORT)");
    return -1;
  }
#endif

  struct sockaddr_in ssin = { 0 };

  ssin.sin_addr.s_addr = htonl(INADDR_ANY);
  ssin.sin_port = htons(1024);
  ssin.sin_family = AF_INET;

  if(bind (ssock, (struct sockaddr*) &ssin, sizeof(ssin)) == -1)
  {
    perror("bind()");
    return -1;
  }

  if(listen(ssock, 1) == -1)
  {
    perror("listen()");
    return -1;
  }

  struct sockaddr_in csin = { 0 };
  socklen_t csz = sizeof(csin);
  int csock = accept(ssock, (struct sockaddr*) &csin, &csz);

  if (csock == -1) {
    perror("accept()");
    return -1;
  }

  for (int i = 1; i < argc; ++i) {
    if (send(csock, argv[i], strlen(argv[i]), 0) == -1) {
      char s[32];

      sprintf(s, "send %i", i);
      perror(s);
    }
  }

  close(csock);
  close(ssock);
}

client.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <errno.h>
#include <string.h>

int main(int argc, char ** argv)
{
  if (argc != 2) {
    printf("Usage : %s <length>\n", *argv);
    return 0;
  }

  int len;
  char c;

  if ((sscanf(argv[1], "%d%c", &len, &c) != 1) && (len < 1)) {
    fprintf(stderr, "invalid length\n");
    return -1;
  }

  errno = 0;

  int sock = socket(AF_INET, SOCK_STREAM, 0);

  if (sock == -1) {
    perror("socket()");
    return -1;
  }

  struct sockaddr_in sin = { 0 };

  sin.sin_addr.s_addr = htonl(0x7f000001); /* 127.0.0.1 */
  sin.sin_port = htons(1024);
  sin.sin_family = AF_INET;

  if(connect (sock, (struct sockaddr*) &sin, sizeof(sin)) == -1)
  {
    perror("connect()");
    return -1;
  }

  for (;;) {
    size_t ln = len;
    char * buf = malloc(ln + 1);

    if (buf == NULL) {
      fprintf(stderr, "cannot malloc");
      break;
    }

    size_t off = 0;

    for (;;) {
      ssize_t sz = recv(sock, buf + off, len, MSG_PEEK); /* no MSG_TRUNC : AF_INET */

      if (sz <= 0) {
        free(buf);
        close(sock);
        return -1;
      }

      buf[off + sz] = 0;

      /* debug */
      printf("peek '%s'\n", buf + off);

      char * p = strchr(buf + off, ' ');

      if (p != NULL) {
        recv(sock, buf + off, p - buf - off + 1, 0);
        *p = 0;
        printf("full buff is '%s'\n", buf);
        free(buf);
        break;
      }

      recv(sock, buf + off, sz, 0);
      off += sz;
      ln += sz;
      buf = realloc(buf, ln + 1);

      if (buf == NULL) {
        fprintf(stderr, "cannot malloc");
        break;
      }
    }
  }

  close(sock);
}

Compilation and execution :

pi@raspberrypi:~ $ gcc -pedantic -Wextra server.c -o se
pi@raspberrypi:~ $ gcc -g -pedantic -Wextra client.c -o cl
pi@raspberrypi:~ $ ./se "123 456 78901234567" "8 1 " &
[1] 11551
pi@raspberrypi:~ $ ./cl 5
peek '123 4'
full buff is '123'
peek '456 7'
full buff is '456'
peek '78901'
peek '23456'
peek '78 1 '
full buff is '789012345678'
peek '1 '
full buff is '1'
[1]+  Fini                    ./se "123 456 78901234567" "8 1 "
pi@raspberrypi:~ $ 

Executions under valgrind (in separate terminals) :

pi@raspberrypi:~ $ valgrind ./se "123 456 78901234567" "8 1 " 
==11602== Memcheck, a memory error detector
==11602== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==11602== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==11602== Command: ./se 123\ 456\ 78901234567 8\ 1\ 
==11602== 
==11602== 
==11602== HEAP SUMMARY:
==11602==     in use at exit: 0 bytes in 0 blocks
==11602==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==11602== 
==11602== All heap blocks were freed -- no leaks are possible
==11602== 
==11602== For counts of detected and suppressed errors, rerun with: -v
==11602== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 3)

pi@raspberrypi:~ $ valgrind ./cl 5
==11604== Memcheck, a memory error detector
==11604== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==11604== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==11604== Command: ./cl 5
==11604== 
peek '123 4'
full buff is '123'
peek '456 7'
full buff is '456'
peek '78901'
peek '23456'
peek '78 1 '
full buff is '789012345678'
peek '1 '
full buff is '1'
==11604== 
==11604== HEAP SUMMARY:
==11604==     in use at exit: 0 bytes in 0 blocks
==11604==   total heap usage: 8 allocs, 8 frees, 1,081 bytes allocated
==11604== 
==11604== All heap blocks were freed -- no leaks are possible
==11604== 
==11604== For counts of detected and suppressed errors, rerun with: -v
==11604== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 3)

Answer 2

You don't "put it back". Instead, define what constitutes a complete message and implement a buffer the reads fixed amounts from the socket and extracts only complete messages.

For example, the class below would call recv until a sentinel byte (in this case, newline) is found, then return just the message (a UTF-8-encoded string) minus the sentinel. Any remaining data in the buffer is saved and processed at the next get_msg call:

from socket import *

class SocketBuffer:
    def __init__(self,sock):
        self.sock = sock
        self.buffer = b''

    def get_msg(self):
        # Buffer data until a newline is found.
        while b'\n' not in self.buffer:
            data = self.sock.recv(1024)
            if not data:
                return b'' # drops partial messages...should check and raise error instead
            self.buffer += data
        # split off the message bytes from the buffer.
        msg,_,self.buffer = self.buffer.partition(b'\n')
        return msg.decode()

Answer 3

If you have a fixed size you want to receive you could do something like this:

ssize_t recv_all(int socket, char *buffer_ptr, size_t bytes_to_recv)
{
    size_t original_bytes_to_recv = bytes_to_recv;

    // Continue looping while there are still bytes to receive
    while (bytes_to_recv > 0)
    {
        ssize_t ret = recv(socket, buffer_ptr, bytes_to_recv, 0);
        if (ret <= 0)
        {
            // Error or connection closed
            return ret;
        }

        // We have received ret bytes
        bytes_to_recv -= ret;  // Decrease size to receive for next iteration
        buffer_ptr += ret;     // Increase pointer to point to the next part of the buffer
    }

    return original_bytes_to_recv;  // Now all data have been received
}

Simply use as

// Somewhere above we have received the size of the data to receive...

// Our data buffer
char buffer[the_full_size_of_data];

// Receive all data
recv_all(socket, buffer, sizeof buffer);  // TODO: Add error checking

_{[Note that I use POSIX types like ssize_t and int for the sockets. Modify to fit your system (e.g. SOCKET for the socket on Windows).]}