CODEWITHSUNDEEP
c++plottimebenchmarking
Daniel Franklin
Daniel Franklin

Reputation: 47

Time measuring in C++ (gettimeofday) giving repetitive results

I am trying to insert multiple items in a hashtable and measure the insertion times in milisseconds. Basically, it works like this (this function belongs to the class of my hashtable):

double benchmark(int amountOfInsertions){
    int valueToInsert;
    timeval tv_timeStart, tv_timeEnd;
    double totalTime = 0;
    double db_timeStart, db_timeEnd;

    for (int i = 0; i < amountOfInsertions; i++){

        valueToInsert = generateRandomVariable();

        gettimeofday(&tv_timeStart, NULL);
        insert(valueToInsert);
        gettimeofday(&tv_timeEnd, NULL);

        db_timeStart = tv_timeStart.tv_sec*1000 + tv_timeStart.tv_usec/1000.0;
        db_timeEnd  = tv_timeEnd.tv_sec*1000  + tv_timeEnd.tv_usec/1000.0;

        totalTime += (db_timeEnd - db_timeStart);
    }

    return totalTime;
}

The problem is that the insertion times used to look like this, obviously showing a clear progression of times, the more items I inserted: enter image description here

But now, I notice that the insertion times kind of alternate between the same values (around multiples of 15.625), creating extremely inaccurate results: enter image description here

And it just started happening all of a sudden, even with old versions of my code that I know output correct times. Is it a particular problem with gettimeofday()? If not, what could it be?

This problem is so mysterious to me that even wonder if this is the right place of right way to ask about it.

UPDATE: I've also tried with clock() and std::chrono::steady_clock, as well as measuring the time of the whole loop instead of each individual insertion (example below), and still got the same behaviour:

double benchmark(int amountOfInsertions){
    int valueToInsert;
    double totalTime = 0;

    steady_clock::time_point t1 = steady_clock::now();
    for (int i = 0; i < amountOfInsertions; i++){

        valueToInsert = generateRandomVariable();
        insert(valueToInsert);

    }
    steady_clock::time_point t2 = steady_clock::now();

    duration<double> time_span = duration_cast<duration<double>>(t2 - t1);
    totalTime = time_span.count()*1000;

    return totalTime;
}

Upvotes: 0

Views: 732

Answers (2)

Marko Bencik
Marko Bencik

Reputation: 396

If you want to truly benchmark this you need to see which optimization flags are you using, is something optimized away, is something running in the background are the context switches from the hyperthreading affecting you and some more. The maybe use Celero or Hayai depends on how precise you need this. Then perform the test at least 5 times and play around with the sample count in the test.

I found out that std chrono is not the most reliable clock if you are benchmarking and trying to define a benchmarking test.

Upvotes: 0

KompjoeFriek
KompjoeFriek

Reputation: 3875

I do not know what caused this sudden change in timer resolution for gettimeofday, but i understand that it should not be used to measure time anyway. Even the man page of gettimeofday says so.

Please use clock_gettime instead. Or if you can use fancy C++11 features: std::chrono::steady_clock

Upvotes: 4

Related Questions