Multicasting via UDP from a different thread

Question

I was trying to create an application that allows me to multicast my webcam feed over my LAN using a specific multicast address and using sendto() to just send the frame buffer. The application I am trying to build is pretty much the same as on this site http://nashruddin.com/Streaming_OpenCV_Videos_Over_the_Network and uses the same architecture. Only instead of a TCP socket I use SOCK_DGRAM. The problem is that when I use the sendto() function from a different thread it tends to fail i.e it returns -1 and errno gets set to 90 (EMSGSIZE), this basically means that the packet formed is too large to be sent over the network. But this happens even if I try to send a simple string (like "hello") to the same multicast address. This seems to work fine if the application is a single thread one. that is to say i just capture the image and multicast it all in the same thread. This is the code:

#include <netinet/in.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
#include "cv.h"
#include "highgui.h"

#define PORT 12345
#define GROUP "225.0.0.37"
CvCapture*  capture;
IplImage*   img0;
IplImage*   img1;
int         is_data_ready = 0;
int         serversock, clientsock;

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

void* streamServer(void* arg);
void  quit(char* msg, int retval);

int main(int argc, char** argv)
{
      pthread_t     thread_s;
int         key;

if (argc == 2) {
    capture = cvCaptureFromFile(argv[1]);
} else {
    capture = cvCaptureFromCAM(0);
}

if (!capture) {
    quit("cvCapture failed", 1);
}

img0 = cvQueryFrame(capture);
img1 = cvCreateImage(cvGetSize(img0), IPL_DEPTH_8U, 1);

cvZero(img1);
cvNamedWindow("stream_server", CV_WINDOW_AUTOSIZE);

/* print the width and height of the frame, needed by the client */
fprintf(stdout, "width:  %d\nheight: %d\n\n", img0->width, img0->height);
fprintf(stdout, "Press 'q' to quit.\n\n");

/* run the streaming server as a separate thread */
if (pthread_create(&thread_s, NULL, streamServer, NULL)) {
    quit("pthread_create failed.", 1);
}

while(key != 'q') {
    /* get a frame from camera */
    img0 = cvQueryFrame(capture);
    if (!img0) break;

    img0->origin = 0;
    cvFlip(img0, img0, -1);

    /**
     * convert to grayscale 
     * note that the grayscaled image is the image to be sent to the client 
     * so we enclose it with pthread_mutex_lock to make it thread safe 
     */
    pthread_mutex_lock(&mutex);
    cvCvtColor(img0, img1, CV_BGR2GRAY);
    is_data_ready = 1;
    pthread_mutex_unlock(&mutex);

    /* also display the video here on server */
    cvShowImage("stream_server", img0);
    key = cvWaitKey(30);
}

/* user has pressed 'q', terminate the streaming server */
if (pthread_cancel(thread_s)) {
    quit("pthread_cancel failed.", 1);
}

/* free memory */
cvDestroyWindow("stream_server");
quit(NULL, 0);
}

/**
 * This is the streaming server, run as a separate thread
 * This function waits for a client to connect, and send the grayscaled images
 */ 
void* streamServer(void* arg)
{
struct sockaddr_in server;

/* make this thread cancellable using pthread_cancel() */
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);

/* open socket */
if ((serversock = socket(AF_INET, SOCK_DGRAM, 0)) == -1) {
    quit("socket() failed", 1);
}
memset(&server,0,sizeof(server));
server.sin_family = AF_INET;
server.sin_port = htons(PORT);
server.sin_addr.s_addr = inet_addr(GROUP);
int opt = 1;
//if(setsockopt(serversock,SOL_SOCKET,SO_BROADCAST,&opt,sizeof(int))==-1){
  //  quit("setsockopt failed",0);
//}
//  /* setup server's IP and port */
//  memset(&server, 0, sizeof(server));
//  server.sin_family = AF_INET;
//  server.sin_port = htons(PORT);
//  server.sin_addr.s_addr = INADDR_ANY;
//   
//  /* bind the socket */
//  if (bind(serversock, (const void*)&server, sizeof(server)) == -1) {
//      quit("bind() failed", 1);
//  }
//
//  /* wait for connection */
//  if (listen(serversock, 10) == -1) {
//      quit("listen() failed.", 1);
//  }
//
//  /* accept a client */
//  if ((clientsock = accept(serversock, NULL, NULL)) == -1) {
//      quit("accept() failed", 1);
//  }

/* the size of the data to be sent */
int imgsize = img1->imageSize;
int bytes=0, i;

/* start sending images */
while(1) 
{
    /* send the grayscaled frame, thread safe */
    pthread_mutex_lock(&mutex);
    if (is_data_ready) {
//          bytes = send(clientsock, img1->imageData, imgsize, 0);
        is_data_ready = 0;
        if((bytes = sendto(serversock,img1->imageData,imgsize,0,(struct sockaddr*)&server,sizeof(server)))==-1){
            quit("sendto FAILED",1);
        }
    }
    pthread_mutex_unlock(&mutex);

//      /* if something went wrong, restart the connection */
//      if (bytes != imgsize) {
//          fprintf(stderr, "Connection closed.\n");
//          close(clientsock);
//
//          if ((clientsock = accept(serversock, NULL, NULL)) == -1) {
//              quit("accept() failed", 1);
//          }
//      }

    /* have we terminated yet? */
    pthread_testcancel();

    /* no, take a rest for a while */
    usleep(1000);
}
}

 /**
  * this function provides a way to exit nicely from the system
  */
void quit(char* msg, int retval)
 {
    if (retval == 0) {
    fprintf(stdout, (msg == NULL ? "" : msg));
    fprintf(stdout, "\n");
} else {
    fprintf(stderr, (msg == NULL ? "" : msg));
    fprintf(stderr, "\n");
}

if (clientsock) close(clientsock);
if (serversock) close(serversock);
if (capture) cvReleaseCapture(&capture);
if (img1) cvReleaseImage(&img1);

pthread_mutex_destroy(&mutex);

    exit(retval);
}

Answer 1

In the sendto() call, you reference imgsize which is initialized to img1->imageSize. But I don't see where img1->imageSize is set and it appears that imgsize is never updated.

So first check that the imgsize value being passed to sendto() is correct. Then check that it is not too large:

UDP/IP datagrams have a hard payload limit of 65,507 bytes. However, an IPv4 network is not required to support more than 548 bytes of payload. (576 is the minimum IPv4 MTU size, less 28 bytes of UDP/IP overhead). Most networks have an MTU of 1500, giving you a nominal payload of 1472 bytes.

Most networks allow you to exceed the MTU by breaking the datagram into IP fragments, which the receiving OS must reassemble. This is invisible to your application: recvfrom() either gets the whole reassembled packet or it gets nothing. But the odds of getting nothing go up with fragmentation because the loss of any fragment will cause the entire packet to be loss. In addition, some routers and operating systems have obscure security rules which will block some UDP patterns or fragments of certain sizes.

Finally, any given network may enforce a maximum datagram size even with fragmentation, and this is often much less than 65507 bytes.

Since you are dealing with a specific network, you will need to experiment to see how big you can reliably go.

UDP/IP at Wikipedia

IPv4 at Wikipedia

Answer 2

Are you absolutely sure that you don't try to send more than limit of UDP which is around 65500 bytes? From my experience you shouldn't even send more than Ethernet packet limit which is around 1500 bytes to keep best UDP reliability.

I think that right now you are trying to send much more data in a form of stream. UDP isn't a stream protocol and you can't replace TCP by it. But of course it is possible to use UDP to send video stream on multicast, but you need some protocol on top of UDP that will handle message size limit of UDP. In real world RTP protocol on top of UDP is used for such kind of task.