Kilobytes or Kibibytes in GNU time?

Question

We are doing some performance measurements including some memory footprint measurements. We've been doing this with GNU time. But, I cannot tell if they are measuring in kilobytes (1000 bytes) or kibibytes (1024 bytes).

The man page for my system says of the %M format key (which we are using to measure peak memory usage): "Maximum resident set size of the process during its lifetime, in Kbytes."

I assume K here means the SI "Kilo" prefix, and thus kilobytes. But having looked at a few other memory measurements of various things through various tools, I trust that assumption like I'd trust a starved lion to watch my dogs during a week-long vacation.

I need to know, because for our tests 1000 vs 1024 Kbytes adds up to a difference of nearly 8 gigabytes, and I'd like to think I can cut down the potential error in our measurements by a few billion.

Answer 1

Using the below testing setup, I have determined that GNU time on my system measures in Kibibytes.

The below program (allocator.c) allocates data and touches each of it 1 KiB at a time to ensure that it all gets paged in. Note: This test only works if you can page in the entirety of the allocated data, otherwise time's measurement will only be the largest resident collection of memory.

allocator.c:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define min(a,b) ( ( (a)>(b) )? (b) : (a) )

volatile char access;
volatile char* data;

const int step = 128;

int main(int argc, char** argv ){

  unsigned long k = strtoul( argv[1], NULL, 10 );
  if( k >= 0 ){
    printf( "Allocating %lu (%s) bytes\n", k, argv[1] );
    data = (char*) malloc( k );

    for( int i = 0; i < k; i += step ){
      data[min(i,k-1)] = (char) i;
    }

    free( data );
  } else {
    printf("Bad size: %s => %lu\n", argv[1], k ); 
  }
  return 0;
}

compile with: gcc -O3 allocator.c -o allocator

Runner Bash Script:

kibibyte=1024
kilobyte=1000
mebibyte=$(expr 1024 \* ${kibibyte})
megabyte=$(expr 1000 \* ${kilobyte})
gibibyte=$(expr 1024 \* ${mebibyte})
gigabyte=$(expr 1000 \* ${megabyte})

for mult in $(seq 1 3);
do
  bytes=$(expr ${gibibyte} \* ${mult} )
  echo ${mult} GiB \(${bytes} bytes\)
  echo "... in kibibytes: $(expr ${bytes} / ${kibibyte})"
  echo "... in kilobytes: $(expr ${bytes} / ${kilobyte})"
  /usr/bin/time -v ./allocator ${bytes}
  echo "===================================================="
done

For me this produces the following output:

1 GiB (1073741824 bytes)
... in kibibytes: 1048576
... in kilobytes: 1073741
Allocating 1073741824 (1073741824) bytes
    Command being timed: "./a.out 1073741824"
    User time (seconds): 0.12
    System time (seconds): 0.52
    Percent of CPU this job got: 75%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.86
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 1049068
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 0
    Minor (reclaiming a frame) page faults: 262309
    Voluntary context switches: 7
    Involuntary context switches: 2
    Swaps: 0
    File system inputs: 16
    File system outputs: 8
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0
====================================================
2 GiB (2147483648 bytes)
... in kibibytes: 2097152
... in kilobytes: 2147483
Allocating 2147483648 (2147483648) bytes
    Command being timed: "./a.out 2147483648"
    User time (seconds): 0.21
    System time (seconds): 1.09
    Percent of CPU this job got: 99%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.31
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 2097644
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 0
    Minor (reclaiming a frame) page faults: 524453
    Voluntary context switches: 4
    Involuntary context switches: 3
    Swaps: 0
    File system inputs: 0
    File system outputs: 8
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0
====================================================
3 GiB (3221225472 bytes)
... in kibibytes: 3145728
... in kilobytes: 3221225
Allocating 3221225472 (3221225472) bytes
    Command being timed: "./a.out 3221225472"
    User time (seconds): 0.38
    System time (seconds): 1.60
    Percent of CPU this job got: 99%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.98
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 3146220
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 0
    Minor (reclaiming a frame) page faults: 786597
    Voluntary context switches: 4
    Involuntary context switches: 3
    Swaps: 0
    File system inputs: 0
    File system outputs: 8
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0
====================================================

In the "Maximum resident set size" entry, I see values that are closest to the kibibytes value I expect from that raw byte count. There is some difference because its possible that some memory is being paged out (in cases where it is lower, which none of them are here) and because there is more memory being consumed than what the program allocates (namely, the stack and the actual binary image itself).

Versions on my system:

> gcc --version
gcc (GCC) 6.1.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

> /usr/bin/time --version
GNU time 1.7

> lsb_release -a
LSB Version:    :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description:    CentOS release 6.10 (Final)
Release:    6.10
Codename:   Final