CODEWITHSUNDEEP
linuxbashprocessiodisk
Wajih
Wajih

Reputation: 659

Calculate Total disk i/o by a single process

I am looking for some tool that will dump total disk I/O by a single process after it ends. So far my finding is :-

For example, I have some process running in background with PID ####. I need the Total Bytes Written and Read by that process in total after the process ends.Can anybody tell how I can extract this information given a process PID.

Upvotes: 7

Views: 3918

Answers (2)

thatsafunnyname
thatsafunnyname

Reputation: 176

This approach works by storing the shell process io counters from /proc/[pid]/io/ in an environment variable before running the process, and then subtracting them after the process ends.

For /proc/[pid]/io counter definitions see:

In this example there are no read_bytes or write_bytes from/to storage:

> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io

> head -c 100M < /dev/urandom > /dev/null

> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
                rchar: 104863647
                wchar: 104858043
                syscr: 12941
                syscw: 25608
           read_bytes: 0
          write_bytes: 0
cancelled_write_bytes: 0

In this example there are write_bytes to storage:

> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io

> head -c 100M < /dev/urandom > /tmp/test.tmp

> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
                rchar: 104863684
                wchar: 104857948
                syscr: 12978
                syscw: 25621
           read_bytes: 0
          write_bytes: 104857600
cancelled_write_bytes: 0

In this example there are no read_bytes or write_bytes from/to storage:

> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io

> cat /tmp/test.tmp > /dev/null

> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
                rchar: 104863564
                wchar: 104858183
                syscr: 859
                syscw: 836
           read_bytes: 0
          write_bytes: 0
cancelled_write_bytes: 0

There are 0 read_bytes because /tmp/test.tmp is all in memory(in core).

> fincore /tmp/test.tmp
  RES PAGES  SIZE FILE
 100M 25600  100M /tmp/test.tmp

dd can be used to perform direct IO from the input file, so now there are read_bytes from storage:

> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io

> dd if=/tmp/test.tmp of=/dev/null iflag=direct
204800+0 records in
204800+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 5.77554 s, 18.2 MB/s

> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
                rchar: 104863573
                wchar: 104858200
                syscr: 204868
                syscw: 204852
           read_bytes: 104857600
          write_bytes: 0
cancelled_write_bytes: 0

dd can be used to perform direct IO from the input file and to the output file:

> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io

> dd if=/tmp/test.tmp of=/tmp/test2.tmp iflag=direct oflag=direct
204800+0 records in
204800+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 11.9083 s, 8.8 MB/s

> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
                rchar: 104863754
                wchar: 104858310
                syscr: 205049
                syscw: 204863
           read_bytes: 104857600
          write_bytes: 104857600
cancelled_write_bytes: 0

> rm /tmp/test.tmp /tmp/test2.tmp

Upvotes: 0

Cyrus
Cyrus

Reputation: 88601

Feel free to play with this scribble (myio.sh):

#!/bin/bash 

TEMPFILE=$(tempfile)    # create temp file for results

trap "rm $TEMPFILE; exit 1" SIGINT  # cleanup after Ctrl+C

SECONDS=0               # reset timer

$@ &                    # execute command in background

IO=/proc/$!/io          # io data of command
while [ -e $IO ]; do
    cat $IO > "$TEMPFILE"   # "copy" data
    sed 's/.*/& Bytes/' "$TEMPFILE" | column -t
    echo
    sleep 1
done

S=$SECONDS              # save timer

echo -e "\nPerformace after $S seconds:"
while IFS=" " read string value; do
    echo $string $(($value/1024/1024/$S)) MByte/s
done < "$TEMPFILE" | column -t

rm "$TEMPFILE"          # remove temp file

Syntax: ./myio.sh <your command>

Examples:

  • ./myio.sh dd if=/dev/zero of=/dev/null bs=1G count=4096
  • as root: ./myio.sh dd if=/dev/sda1 of=/dev/null bs=1M count=4096

Please change dd's of= in last example only if you know what you are doing.

With this simple script from me you can watch an already running process and its IO.

Syntax: pio.sh PID

#!/bin/bash

[ "$1" == "" ] && echo "Error: Missing PID" && exit 1
IO=/proc/$1/io          # io data of PID
[ ! -e "$IO" ] && echo "Error: PID does not exist" && exit 2
I=3                     # interval in seconds
SECONDS=0               # reset timer

echo "Watching command $(cat /proc/$1/comm) with PID $1"

IFS=" " read rchar wchar syscr syscw rbytes wbytes cwbytes < <(cut -d " " -f2 $IO | tr "\n" " ")

while [ -e $IO ]; do
    IFS=" " read rchart wchart syscrt syscwt rbytest wbytest cwbytest < <(cut -d " " -f2 $IO | tr "\n" " ")

    S=$SECONDS
    [ $S -eq 0 ] && continue

cat << EOF
rchar:                 $((($rchart-$rchar)/1024/1024/$S)) MByte/s
wchar:                 $((($wchart-$wchar)/1024/1024/$S)) MByte/s
syscr:                 $((($syscrt-$syscr)/1024/1024/$S)) MByte/s
syscw:                 $((($syscwt-$syscw)/1024/1024/$S)) MByte/s
read_bytes:            $((($rbytest-$rbytes)/1024/1024/$S)) MByte/s
write_bytes:           $((($wbytest-$wbytest)/1024/1024/$S)) MByte/s
cancelled_write_bytes: $((($cwbytest-$cwbytes)/1024/1024/$S)) MByte/s
EOF
    echo
    sleep $I
done

Upvotes: 6

Related Questions