How to send and receive an array of integers from client to server with C sockets?

Question

The problem: send and receive an array of integers (later floats) from client to server using TCP and the sockets C API. Must run both in Winsock and UNIX.

In the future different endianess for client/server can be handled, but now the test was made between 2 machines with same endianess (Windows client, Linux server).

I implemented the client and server, all seems to work, but the question is a doubt on how the send() (client) and recv() (server) calls handle the way my implementation is made. Or, in other words, if the approach has a flaw.

The approach was:

1. On the client generate a vector of uint8_t according to a predefined algorithm (N sequences of values from 0 to 254). This sequence is reproduced in the server to compare with the incoming data (by comparing 2 vectors).

2. On the client, send the array size

3. On the client, send the array using a loop on the array, call send() for each element.

4. On the server, recv() the array size.

5. On the server, recv() the array using a loop on the array size, call recv() for each element.

To check my approach,

6. I save the bytes received on the server to a file inside the previous recv() loop

7. After the loop , read this file, generate another vector with the same size according to step 1), compare the 2 vectors. They match, using tests up 255,000,000 array elements sent and received.

Question: One can then assume that the server recv() loop is guaranteed to match the client send() loop?

Or, in other words, that the array indices arrive in the same order?

I am following the excellent "TCP/IP Sockets in C" (Donahoo, Calvert) and on the example of echo client / server

http://cs.baylor.edu/~donahoo/practical/CSockets/

Quote:

"The bytes sent by a call to send() on one end of a connection may not all be returned by a single call to recv() on the other end."

The recv() part is handled differently in this example, a loop is made until the total number of bytes received matches the (known size) of bytes sent, according to:

while (totalBytesRcvd < echoStringLen)
{
  bytesRcvd = recv(sock, echoBuffer, RCVBUFSIZE - 1, 0))
  totalBytesRcvd += bytesRcvd;   /* Keep tally of total bytes */
}

Complete example:

http://cs.baylor.edu/~donahoo/practical/CSockets/code/TCPEchoClient.c

But this case is for one send() with multiple bytes call that might be not received all at once.

In my case there are N send calls (1 byte each) and N receive calls (1 byte each), that happen to be made in the same order.

Question: Does the TCP/IP protocol guarantee that the multiple send calls (that have sequential time stamps) are guaranteed to be received in order? Or time not an issue here?

Some research:

When sending an array of int over TCP, why are only the first amount correct?

Quote:

"There is nothing to guarantee how TCP will packet up the data you send to a stream - it only guarantees that it will end up in the correct order at the application level."

Some more links

How do I send an array of integers over TCP in C?

Thanks

EDIT : code edited with main() functions and usage, and variable names for clarity

Usage example: send N sequences 1 time to server at IP-address

./client -i IP-address -n N -d

Code: Client.cpp

#if defined (_MSC_VER)
#include <winsock.h>
#else
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#endif

#include <iostream>
#include <stdint.h>
#include <vector>

const unsigned short server_port = 5000;  // server port
void client_echo_text(const char *server_ip);
void client_send_data(const char *server_ip, const uint32_t arr_size_mult);

///////////////////////////////////////////////////////////////////////////////////////
//main
///////////////////////////////////////////////////////////////////////////////////////

// usage: 
// send N sequences 1 time to server at <IP adress>
// ./client -i <IP adress> -n N -d 
// same with infinite loop
// ./client -i <IP adress> -n N -l 

int main(int argc, char *argv[])
{
  char server_ip[255]; // server IP address (dotted quad)
  strcpy(server_ip, "127.0.0.1");
  uint32_t arr_size_mult = 10;

  //no arguments
  if (argc == 1)
  {
    client_send_data(server_ip, arr_size_mult);
  }

  for (int i = 1; i < argc && argv[i][0] == '-'; i++)
  {
    switch (argv[i][1])
    {
    case 'i':
      strcpy(server_ip, argv[i + 1]);
      i++;
      break;
    case 'e':
      client_echo_text(server_ip);
      exit(0);
      break;
    case 'n':
      arr_size_mult = atoi(argv[i + 1]);
      i++;
      break;
    case 'd':
      client_send_data(server_ip, arr_size_mult);
      exit(0);
      break;
    case 'l':
      while (true)
      {
        client_send_data(server_ip, arr_size_mult);
      }
      break;
    }
  }

  return 0;
}

///////////////////////////////////////////////////////////////////////////////////////
//client_send_data
///////////////////////////////////////////////////////////////////////////////////////

void client_send_data(const char *server_ip, const uint32_t arr_size_mult)
{
  int sock;                          // socket descriptor
  struct sockaddr_in server_addr;    // server address

  //data
  const uint32_t arr_size = arr_size_mult * 255; // array size

  //construct array
  std::vector<uint8_t> val8(arr_size);
  uint8_t v8 = 0;
  for (size_t i = 0; i < arr_size; ++i)
  {
    val8[i] = v8;
    v8++;
    if (v8 == 255)
    {
      v8 = 0;
    }
  }

#if defined (_MSC_VER)
  WSADATA ws_data;
  if (WSAStartup(MAKEWORD(2, 0), &ws_data) != 0)
  {
    exit(1);
  }
#endif

  // create a stream socket using TCP
  if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
  {
    exit(1);
  }

  // construct the server address structure
  memset(&server_addr, 0, sizeof(server_addr));          // zero out structure
  server_addr.sin_family = AF_INET;                      // internet address family
  server_addr.sin_addr.s_addr = inet_addr(server_ip);    // server IP address
  server_addr.sin_port = htons(server_port);             // server port

  // establish the connection to the server
  if (connect(sock, (struct sockaddr *) &server_addr, sizeof(server_addr)) < 0)
  {
    std::cout << "connect error: " << strerror(errno) << std::endl;
    exit(1);
  }

  //send array size
  if (send(sock, (char *)&arr_size, sizeof(uint32_t), 0) != sizeof(uint32_t))
  {
    exit(1);
  }

  std::cout << "client sent array size: " << (int)arr_size << std::endl;

  //send array
  for (size_t i = 0; i < arr_size; ++i)
  {
    v8 = val8[i];
    if (send(sock, (char *)&v8, sizeof(uint8_t), 0) != sizeof(uint8_t))
    {
      exit(1);
    }
  }

  std::cout << "client sent array: " << std::endl;

#if defined (_MSC_VER)
  closesocket(sock);
  WSACleanup();
#else
  close(sock);
#endif

}

Code: Server.cpp

if defined(_MSC_VER)
#include <winsock.h>
#else
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#endif

#include <iostream>
#include <stdint.h>
#include <assert.h>
#include <vector>

const unsigned short server_port = 5000;  // server port
void server_echo_text();
void server_recv_data(bool verbose);
void check_file(const uint32_t arr_size, bool verbose, const size_t slab_size);

///////////////////////////////////////////////////////////////////////////////////////
//main
///////////////////////////////////////////////////////////////////////////////////////

int main(int argc, char *argv[])
{
  bool verbose = false;

  //no arguments
  if (argc == 1)
  {
    server_recv_data(verbose);
  }

  for (int i = 1; i < argc && argv[i][0] == '-'; i++)
  {
    switch (argv[i][1])
    {
    case 'v':
      std::cout << "verbose mode: " << std::endl;
      verbose = true;
      break;
    case 'e':
      std::cout << "running echo server: " << std::endl;
      server_echo_text();
      exit(0);
      break;
    case 'd':
      std::cout << "running data server: " << std::endl;
      server_recv_data(verbose);
      exit(0);
      break;
    }
  }

  return 0;

}

///////////////////////////////////////////////////////////////////////////////////////
//server_recv_data
///////////////////////////////////////////////////////////////////////////////////////

void server_recv_data(bool verbose)
{
  const int MAXPENDING = 5;             // maximum outstanding connection requests
  int server_socket;                    // socket descriptor for server
  int client_socket;                    // socket descriptor for client
  sockaddr_in server_addr;              // local address
  sockaddr_in client_addr;              // client address
  int  recv_size;                       // size in bytes returned by recv() 
#if defined (_MSC_VER)
  int len_addr;                         // length of client address data structure
#else
  socklen_t len_addr;
#endif

  //data
  uint32_t arr_size = 0;
  size_t slab_size = 1;
  FILE *file;

#if defined (_MSC_VER)
  WSADATA ws_data;
  if (WSAStartup(MAKEWORD(2, 0), &ws_data) != 0)
  {
    exit(1);
  }
#endif

  // create socket for incoming connections
  if ((server_socket = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
  {
    exit(1);
  }

  // construct local address structure
  memset(&server_addr, 0, sizeof(server_addr));     // zero out structure
  server_addr.sin_family = AF_INET;                 // internet address family
  server_addr.sin_addr.s_addr = htonl(INADDR_ANY);  // any incoming interface
  server_addr.sin_port = htons(server_port);        // local port

  // bind to the local address
  if (bind(server_socket, (sockaddr*)&server_addr, sizeof(server_addr)) < 0)
  {
    //bind error: Permission denied
    //You're probably trying to bind a port under 1024. These ports usually require root privileges to be bound.
    std::cout << "bind error: " << strerror(errno) << std::endl;
    exit(1);
  }

  // mark the socket so it will listen for incoming connections
  if (listen(server_socket, MAXPENDING) < 0)
  {
    exit(1);
  }

  for (;;) // run forever
  {
    // set length of client address structure (in-out parameter)
    len_addr = sizeof(client_addr);

    // wait for a client to connect
    if ((client_socket = accept(server_socket, (struct sockaddr *) &client_addr, &len_addr)) < 0)
    {
      exit(1);
    }

    // convert IP addresses from a dots-and-number string to a struct in_addr and back
    char *str_ip = inet_ntoa(client_addr.sin_addr);
    std::cout << "handling client " << str_ip << std::endl;

    // receive array size
    if ((recv_size = recv(client_socket, (char *)&arr_size, sizeof(uint32_t), 0)) != sizeof(uint32_t))
    {
      exit(1);
    }

    std::cout << "server received array size: " << (int)arr_size << std::endl;

    //save file
    file = fopen("file.bin", "wb");
    fwrite(&arr_size, sizeof(uint32_t), 1, file);

    //receive array
    for (size_t i = 0; i < arr_size; ++i)
    {
      uint8_t v8;
      if ((recv_size = recv(client_socket, (char *)&v8, sizeof(uint8_t), 0)) != sizeof(uint8_t))
      {
        exit(1);
      }

      //write 1 element
      fwrite(&v8, sizeof(uint8_t), slab_size, file);
    }

    fclose(file);

    std::cout << "server received array: " << std::endl;
    check_file(arr_size, verbose, slab_size);

    // close client socket
#if defined (_MSC_VER)
    closesocket(client_socket);
#else
    close(client_socket);
#endif
  }


}

///////////////////////////////////////////////////////////////////////////////////////
//check_file
///////////////////////////////////////////////////////////////////////////////////////

void check_file(const uint32_t arr_size, bool verbose, const size_t slab_size)
{
  //read file
  std::vector<uint8_t> val8(arr_size);
  std::vector<uint8_t> val8_c(arr_size);
  uint32_t arr_size_r;
  uint8_t v8;
  FILE *file;

  file = fopen("file.bin", "rb");
  fread(&arr_size_r, sizeof(uint32_t), 1, file);
  assert(arr_size_r == arr_size);

  for (size_t i = 0; i < arr_size; ++i)
  {
    fread(&v8, sizeof(uint8_t), slab_size, file);
    val8[i] = v8;
    if (verbose) std::cout << (int)val8[i] << " ";

  }
  if (verbose) std::cout << std::endl;

  fclose(file);

  //check data, define array the same as in client, compare arrays
  v8 = 0;
  for (size_t i = 0; i < arr_size; ++i)
  {
    val8_c[i] = v8;
    v8++;
    if (v8 == 255)
    {
      v8 = 0;
    }
  }

  //compare arrays
  for (size_t i = 0; i < arr_size; ++i)
  {
    if (val8_c[i] != val8[i])
    {
      std::cout << "arrays differ at: " << i << " " << (int)val8_c[i] << " " << (int)val8[i] << std::endl;
      assert(0);
    }
  }

  std::cout << "arrays match: " << (int)arr_size << " " << (int)arr_size_r << std::endl;
  std::cout << std::endl;

}

Answer 1

As @usr pointed out, the loops are badly constructed. What is needed are "send all" and "receive all" functions.

These ones are based on the book by Stevens "UNIX Network Programming: Sockets Introduction"

http://www.informit.com/articles/article.aspx?p=169505&seqNum=9

Send all function and send function from client:

void send_all(int sock, const void *vbuf, size_t size_buf)
{
  const char *buf = (char*)vbuf;    // can't do pointer arithmetic on void* 
  int send_size;                  // size in bytes sent or -1 on error 
  size_t size_left;               // size left to send 
  const int flags = 0;

  size_left = size_buf;

  while (size_left > 0)
  {
    if ((send_size = send(sock, buf, size_left, flags)) == -1)
    {
      std::cout << "send error: " << strerror(errno) << std::endl;
      exit(1);
    }

    if (send_size == 0)
    {
      std::cout << "all bytes sent " << std::endl;
      break;
    }

    size_left -= send_size;
    buf += send_size;
  }

  return;
}

 ///////////////////////////////////////////////////////////////////////////////////////
//client_send_data
///////////////////////////////////////////////////////////////////////////////////////

void client_send_data(const char *server_ip, const uint32_t arr_size_mult)
{
  int sock;                          // socket descriptor
  struct sockaddr_in server_addr;    // server address

  //data
  const uint32_t arr_size = arr_size_mult * 255; // array size

  //construct array
  std::vector<uint8_t> val8(arr_size);
  uint8_t v8 = 0;
  for (size_t i = 0; i < arr_size; ++i)
  {
    val8[i] = v8;
    v8++;
    if (v8 == 255)
    {
      v8 = 0;
    }
  }

#if defined (_MSC_VER)
      WSADATA ws_data;
      if (WSAStartup(MAKEWORD(2, 0), &ws_data) != 0)
      {
        exit(1);
      }
#endif

  // create a stream socket using TCP
  if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
  {
    exit(1);
  }

  // construct the server address structure
  memset(&server_addr, 0, sizeof(server_addr));          // zero out structure
  server_addr.sin_family = AF_INET;                      // internet address family
  server_addr.sin_addr.s_addr = inet_addr(server_ip);    // server IP address
  server_addr.sin_port = htons(server_port);             // server port

  // establish the connection to the server
  if (connect(sock, (struct sockaddr *) &server_addr, sizeof(server_addr)) < 0)
  {
    std::cout << "connect error: " << strerror(errno) << std::endl;
    exit(1);
  }

  //send array size
  send_all(sock, (void *)&arr_size, sizeof(uint32_t));

  std::cout << "client sent array size: " << (int)arr_size << std::endl;

  //send array
  //std::vector.data() returns the address of the initial element in the container (C++11)
  send_all(sock, (void *)val8.data(), sizeof(uint8_t) * val8.size());

  std::cout << "client sent array: " << std::endl;

#if defined (_MSC_VER)
      closesocket(sock);
      WSACleanup();
#else
      close(sock);
#endif

}

Receive all function

void recv_all(int sock, void *vbuf, size_t size_buf, FILE *file)
{
  char *buf = (char*)vbuf;  // can't do pointer arithmetic on void* 
  int recv_size;            // size in bytes received or -1 on error 
  size_t size_left;         // size left to send 
  const int flags = 0;

  size_left = size_buf;

  while (size_left > 0)
  {
    if ((recv_size = recv(sock, buf, size_left, flags)) == -1)
    {
      std::cout << "recv error: " << strerror(errno) << std::endl;
      exit(1);
    }

    if (recv_size == 0)
    {
      std::cout << "all bytes received " << std::endl;
      break;
    }

    //save to local file
    fwrite(buf, recv_size, 1, file);

    size_left -= recv_size;
    buf += recv_size;
  }

  return;
}

///////////////////////////////////////////////////////////////////////////////////////
//server_recv_data
///////////////////////////////////////////////////////////////////////////////////////

void server_recv_data(bool verbose)
{
  const int MAXPENDING = 5;             // maximum outstanding connection requests
  int server_socket;                    // socket descriptor for server
  int client_socket;                    // socket descriptor for client
  sockaddr_in server_addr;              // local address
  sockaddr_in client_addr;              // client address
#if defined (_MSC_VER)
      int len_addr;                         // length of client address data structure
#else
      socklen_t len_addr;
#endif

  //data
  uint32_t arr_size = 0;
  const size_t slab_size = 1;
  FILE *file;

#if defined (_MSC_VER)
      WSADATA ws_data;
      if (WSAStartup(MAKEWORD(2, 0), &ws_data) != 0)
      {
        exit(1);
      }
#endif

  // create socket for incoming connections
  if ((server_socket = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
  {
    exit(1);
  }

  // construct local address structure
  memset(&server_addr, 0, sizeof(server_addr));     // zero out structure
  server_addr.sin_family = AF_INET;                 // internet address family
  server_addr.sin_addr.s_addr = htonl(INADDR_ANY);  // any incoming interface
  server_addr.sin_port = htons(server_port);        // local port

  // bind to the local address
  if (bind(server_socket, (sockaddr*)&server_addr, sizeof(server_addr)) < 0)
  {
    //bind error: Permission denied
    //You're probably trying to bind a port under 1024. These ports usually require root privileges to be bound.
    std::cout << "bind error: " << strerror(errno) << std::endl;
    exit(1);
  }

  // mark the socket so it will listen for incoming connections
  if (listen(server_socket, MAXPENDING) < 0)
  {
    exit(1);
  }

  for (;;) // run forever
  {
    // set length of client address structure (in-out parameter)
    len_addr = sizeof(client_addr);

    // wait for a client to connect
    if ((client_socket = accept(server_socket, (struct sockaddr *) &client_addr, &len_addr)) < 0)
    {
      exit(1);
    }

    // convert IP addresses from a dots-and-number string to a struct in_addr and back
    char *str_ip = inet_ntoa(client_addr.sin_addr);
    std::cout << "handling client " << str_ip << std::endl;

    ///////////////////////////////////////////////////////////////////////////////////////
    //receive data and save to local file as received
    ///////////////////////////////////////////////////////////////////////////////////////

    //save local file
    file = fopen(check_file_name.c_str(), "wb");

    //receive/save array size
    recv_all(client_socket, &arr_size, sizeof(uint32_t), file);

    std::cout << "server received array size: " << (int)arr_size << std::endl;

    //receive/save array
    uint8_t *buf = new uint8_t[arr_size];

    recv_all(client_socket, buf, sizeof(uint8_t) * arr_size, file);

    delete[] buf;

    fclose(file);

    std::cout << "server received array: " << std::endl;

    //check
    check_file(arr_size, verbose, slab_size);

    // close client socket
#if defined (_MSC_VER)
        closesocket(client_socket);
#else
        close(client_socket);
#endif
  }


}

Answer 2

TCP is a streaming protocol, it guarantees the exact replication of the sent stream at receiver. So yes, ordering will match, always. The protocol stack will reorder messages if they come out of order. So if you reliably catch the beginning of the stream and the end of the stream then everything in between will come in order and in the good shape.

I am not sure though you'd ever want to send a single number and not pre-marshal them into a large buffer. You will get several orders of magnitude improvement in performance.