CODEWITHSUNDEEP
clinux
MIKU_LINK
MIKU_LINK

Reputation: 192

How to avoid costing too long for writing data to disk

    static const int MAX_BUFFER_LEN = 1024*12; //in byets
    char *bff = new char[MAX_BUFFER_LEN];
    int fileflag = O_CREAT | O_WRONLY | O_NONBLOCK;

    fl = open(filename, fileflag, 0666);

    if(fl < 0)
    {
        printf("can not open file! \n");
        return -1;
    }

    do
    {

        ///begin one loop
        struct timeval bef;
        struct timeval aft;
        gettimeofday(&bef, NULL);

        write(fl, bff, MAX_BUFFER_LEN);

        gettimeofday(&aft, NULL);

        if(aft.tv_usec - bef.tv_usec > 20000) //ignore second condition
        {   
            printf(" cost too long:%d \n", aft.tv_usec - bef.tv_usec);
        }   
        //end one loop

        //sleep
        usleep(30*1000); //sleep 30ms


    }while(1);

When I run the program on Linux ubuntu 2.6.32-24-generic, I find that the COST TOO LONG printing shows 1~2 times in a minutes. I tried both to USB disk and hard disk.I also tried this program in arm platform .This condition also happened. I think that 3.2Mbps is too high for low speed IO device. So I reduce to 0.4Mbps.It significantly reduce the printing frequency. Is any solution to control the time cost ? Is write() just copying the data to kenal buffer and returning immediately or waiting fo disk IO complete? Is it possible that kenal IO buffer is full and must be waiting for flush but why only several times cost so long?

Upvotes: 2

Views: 191

Answers (3)

Nandakumar Karanam
Nandakumar Karanam

Reputation: 36

If you are calling a file write which you think is going to take a lot of time, then make your process to run two threads, while one is doing the main task let the other write to disk.

Upvotes: 1

L. Cornelius Dol
L. Cornelius Dol

Reputation: 64026

Your disk I/O performance is being negatively impacted by the code around each write to measure the time (and measuring time at this granularity is going to have occasional spikes as the computer does other things).

Instead, measure the performance of the code to write the entire data - start/end times outside the loop (with the loop properly bounded, of course).

Upvotes: 2

salezica
salezica

Reputation: 76909

You can't accelerate the disk, but you can do other stuff while the disk is working. You needn't wait for it to be done.

This is, however, highly non-trivial to do in C. You would need nonblocking I/O, multithreading or multiprocessing. Try googling up these keywords and how to use the different techniques (you are already using a nonblocking fd up there).

Upvotes: 3

Related Questions