C Socket Server, memory increases with file writes

Question

I have a simple server program written in C, and the program is running on an Ubuntu Linux distribution. The program is intended to listen for messages sent from the client, write those messages to a file (each message goes into a separate file), and send an acknowledgement back to the client once the message has been received and stored.

I have noticed that, as the server continues to receive and store messages, the amount of available system memory quickly decreases and continues to decrease until messages have stopped. Memory remains constant when no messages are being sent. However, I have also noticed that I can free up the memory again by deleting the written files from disk (I can do this even while the server is still running). I am therefore led to believe that the memory issue has something to do with my file operations, though I can't see any issue with the code that writes the files.

Can someone help?

NOTE: I am observing the memory usage with "top".

I have included an excerpt from the program. The below function handles input from the client and writes that information to file. This is where I currently believe the problem to be:

void handleinput (int sock)
{
    char filename[strlen(tempfolder) + 27];
    generatefilename(filename);

    int rv;
    int n = 1;
    int received = 0;
    char buffer[BUFFER_SIZE];
    FILE *p = NULL;
    fd_set set;
    char response[768];
    struct timeval timeout;
    timeout.tv_sec = 360;
    timeout.tv_usec = 0;

    FD_ZERO(&set);
    FD_SET(sock, &set);

    bzero(buffer, BUFFER_SIZE);
    bzero(response, sizeof response);
    rv = select(sock + 1, &set, NULL, NULL, &timeout);
    if (rv == -1)
    {
        error("error on select in handleinput");
        close(sock);
        exit(1);
    }
    else if (rv == 0)
    {
        close(sock);
        exit(0);
    }
    else
    {
            n = read(sock, buffer, BUFFER_SIZE-1);
            if (n <= 0)
            {
                close(sock);
                exit(0);
            }
    }

    // open file
    if (n != 0)
    {
        p = fopen(filename, "a");
        if (p == NULL)
        {
            error("ERROR writing message to file");
            close(sock);
            exit(1);
        }
    }

    // loop until full message is received
    while (n != 0)
    {
        if (n < 0)
        {
            error("ERROR reading from socket");
            close(sock);
            exit(1);
        }

        received = 1;
        // write content to file
        fwrite(buffer, strlen(buffer), 1, p);

        if (buffer[strlen(buffer)-1] == 0x1c)
        {
            break;
        }

        bzero(buffer, BUFFER_SIZE);
        rv = select(sock + 1, &set, NULL, NULL, &timeout);
        if (rv == -1)
        {
            error("ERROR select in loop in handleinput");
            close(sock);
            exit(1);
        }
        else if (rv == 0)
        {
            close(sock);
            exit(0);
        }
        else
        {
            n = read(sock, buffer, BUFFER_SIZE-1);
        }
    }

    // close file if we opened it earlier
    if (p != NULL)
    {
        fclose(p);
    }

    // send acknowledgement back to client
    if (received == 1)
    {
        generateResponse(response, filename);
        n = write(sock, response, strlen(response));

        if (n < 0)
        {
            error("ERROR writing to socket");
            close(sock);
            exit(1);
        }
    }
}

Answer 1

Its because of the caching mechanism of writing. IF you have a lot of clients trying to write files, the IO buffer fills in kernel memory, but doesn't actually write to that file until you close the socket or the buffer fills. You can fix this just by flushing the buffer. What some other people have suggested is to get rid of the write and read wrappers in stdio, and just use the kernel call write, since this will help performance and will probably flush the buffer automatically.

You can flush with fsync() btw.