CODEWITHSUNDEEP
carrays
amk
amk

Reputation: 340

How to reuse an array to write very fast to a big file in C

I am trying to write fast to a big file and using array. So I have to use same array multiple times. Part of my program is as follows.

    char buff[1024];
    char *x= buff;
    fd = open("file.txt",O_CREAT|O_RDWR, S_IRWXU) ;


    void function(){

        char temp[128];
        sprintf(temp,"%s,%s,%d\n",src,dst,payload) ;
        x=myStrCat(x,temp); // copy files from temp to buff

        //This is my question. if the buffer "buff" is full, how to  truncate the buffer for next loop and that too fast.
        if (strlen(buff) >= 1024){
            write(fd,buff, len);
        }
    }

    char * myStrCat(char * dest, char * src){
             while(*dest) dest++;
             while( *dest++ = *src++);
             return --dest;

     }

    int main(){
        //other codes
        while (true){        
            function();
            //conditions to break the loop untill then function() will be continuously in loop.
        }
        return 0;
    }

Thanks in Advance!

Upvotes: 0

Views: 104

Answers (2)

LPs
LPs

Reputation: 16223

Maybe you are looking for somwthing like this:

    #define BUFFER_SIZE 1024

char buff[BUFFER_SIZE];
uint16_t buff_len = 0;

void function(){

    char temp[128];
    int len = sprintf(temp,"%s,%s,%d\n",src,dst,payload) ;

    if ((len+buff_len) > BUFFER_SIZE-1)
    {
        write(fd,buff, buff_len);

        buff_len = 0;
    }


    strcpy(&buff[buff_len],temp); // copy files from temp to buff

    buff_len += len;
}

Upvotes: 0

Sami Kuhmonen
Sami Kuhmonen

Reputation: 31173

strlen(buf) can never be >= 1024, since you only allocate 1024 bytes for it. C string requires a NULL to be in the end, so you will get a buffer overrun, which leads to undefined behaviour. You could have 1023+NULL, though.

Your code doesn't check if there will be a buffer overrun in the myStrCat either. It will cause undefined behaviour. What if you already have 1020 characters and want to add another 10?

The way you should do this is to keep a number indicating how many characters you already have in the buffer. If the buffer can't hold the next string, write the data into the file and zero the character count. If it can, copy the string to the buffer starting at the position indicated by the character count and get the next one.

Of course in the end write what's in the buffer to the file.

This way you will not run over the buffer limits.

(Are you sure this will be a lot faster than just letting the OS handle the write caching?)

Upvotes: 2

Related Questions