How do I send 10,000 ~ 20,000 bytes data over TCP?

Question

Can I send about 10,000 ~ 20,000 bytes data over TCP? I am transferring an image (60 by 60) from Android client to linux server. On android it seems ok. On server side, if I try to send the picture data back to client, then it doesn't work. On client side, if I parse then I got some weird number that I shouldn't get.

Is there any technical problem of transferring a big data over TCP? How do I fix this? Thanks in advance..

char* PictureResponsePacket::toByte(){

    /*
     * HEADER
     *
     * Magic number (4)
     * Data length  (4)
     * Packet Id    (2)
     * Packet type  (2)
     * Device Id    (48)
     *
     */

    /*
     * BODY
     *
     * Nickname  (48)
     * deviceId  (4)
     * m_pictureSize
     */

    int offset = 0;

    int headerLength = sizeof(int) + sizeof(int) + sizeof(short) + sizeof(short) + 48;
    int bodyLength = 48 + 4 + m_pictureSize;
    int dataLength = headerLength + bodyLength;
    m_dataLength = dataLength;

    log("PictureResponsePacket::toByte(), data length %d \n", m_dataLength);

    char *sendBuffer = new char[dataLength];
    memset(sendBuffer, 0x00, dataLength);

    char *ptr = sendBuffer;

    /*
     * -------------
     * HEADER
     * -------------
     */
    /*
     * Magic number
     */
    memcpy(ptr + offset, m_magicNumberBuffer, sizeof(int));
    offset += sizeof(int);

    /*
     * Data length
     */


    memcpy(ptr + offset, &m_dataLength, sizeof(int));
    offset += sizeof(int);

    /*
     * Packet id
     */
    memcpy(ptr + offset, &m_packetId, sizeof(short));
    offset += sizeof(short);

    /*
     * Packet type
     */
    memcpy(ptr + offset, &m_packetType, sizeof(short));
    offset += sizeof(short);

    /*
     *Device Id
     */

    memcpy(ptr + offset, m_deviceId.c_str(), m_deviceId.size());
    offset += 48;

    /*
     * -------------
     * BODY
     * -------------
     */
    memcpy(ptr + offset, m_senderDeviceId.c_str(), m_senderDeviceId.size());
    offset += 48;

    memcpy(ptr + offset, &m_pictureSize, sizeof(int));
    offset += sizeof(int);

    memcpy(ptr + offset, m_pictureData, m_pictureSize);
    offset += m_pictureSize;


    return sendBuffer;


}

I am getting char* this way and sending it like this

                char * sBuffer = reponsePacket->toByte();
                int remainLength = reponsePacket->getDataLength();
                int currentSentLength = 0;
                SocketClient *client = work->getClient();

                while(remainLength > 0){

                    if(remainLength >= MAX_LENGTH)
                        currentSentLength = send(client->getFd(), sBuffer, MAX_LENGTH, MSG_NOSIGNAL);
                    else
                        currentSentLength = send(client->getFd(), sBuffer, remainLength, MSG_NOSIGNAL);

                    if(currentSentLength == -1){
                        log("WorkHandler::workLoop, connection has been lost \n");
                        break;
                    }

                    sBuffer += currentSentLength;
                    remainLength -= currentSentLength;

Answer 1

Are you saying you can successfully send and read the image if you sent it from Android to your Linux machine? If not then it sounds like it could be an Endian issue, if your Linux Server process was not written in Java.

Update

Since cnicutar has a great answer I'd like to offer an alternative to having to do your protocol manually.

Google Protocol Buffer

When using it, it will automatically convert from host to network and back. It also can be used to create C++ and Java bindings.

Answer 2

What you are trying to do is easy (20K is not "big"). By far the most common reason something like this happens, is disregarding the return codes of send and recv. You should keep a few things in mind:

• send(2) can't always copy all the data from user space to kernel space. Check the returned value
• The data doesn't arrive all at the same time, so you will have to recv a few times before getting all of it

In practice, on many systems you might get away with sending large amounts of data (the kernel will laugh at your 20K) but you will have to receive in a loop. Here is a function heavily inspired by Stevens readn. Use it instead of recv

ssize_t
readn(int fd, void *vptr, size_t n)
{
    size_t nleft;
    ssize_t nread;
    char *ptr;

    ptr = vptr;
    nleft = n;
    while (nleft > 0) {
        if ((nread = read(fd, ptr, nleft)) < 0) {
            if (errno == EINTR)
                /* Loop back and call read again. */
                nread = 0;
            else
                /* Some other error; can't handle. */
                return -1;
        } else if (nread == 0)
            /* EOF. */
            break;

        nleft -= nread;
        ptr += nread;
    }
    return n - nleft;
}

EDIT

You seem to be forgetting about endianness (as Andrew Finnell suspected). For every integer, you should do something like this before sending (before memcpy):

m_dataLength = htonl(m_datalength);

And this when receiving:

m_dataLength = ntohl(m_datalength);

Answer 3

I think strace may help here both on client and server side. Look for network errors and the amount of sent/received data in strace log.