Reputation: 31
I want to run multiple dd commands in background, but be able to see the status.
I have the following script.sh
:
#!/usr/bin/env bash
for drive in $@
do
echo "Wiping $drive"
dd if=/dev/zero of=$drive status=progress &
done
wait
echo "Done."
Which results in the following output:
$ sudo bash ./script.sh /dev/sda /dev/sdb
Wiping /dev/sda
Wiping /dev/sdb
288788992 bytes (289 MB, 275 MiB) copied, 10 s, 28.9 MB/s 14404864 bytes (114 MB, 109 MiB) copied, 4 s, 28.6 MB/s
Is there any way how to output the respective dd statuses below the drive paths? For example:
$ sudo bash ./script.sh /dev/sda /dev/sdb
Wiping /dev/sda
288788992 bytes (289 MB, 275 MiB) copied, 10 s, 28.9 MB/s
Wiping /dev/sdb
14404864 bytes (114 MB, 109 MiB) copied, 4 s, 28.6 MB/s
I tried various redirects, named pipes etc. but wasn't able to achieve such (or similar) output.
I tried the coprocesses approach which seems to be the way to go, but now I'm unable to make it work with the for cycle.
This works fine:
coproc dd_sda { dd if=/dev/zero of=/dev/sda status=progress 2>&1; }
echo "sda PID: $dd_sda_PID"
coproc dd_sdb { dd if=/dev/zero of=/dev/sdb status=progress 2>&1; }
echo "sdb PID: $dd_sdb_PID"
sda PID: 12494
./wipe.sh: line 86: warning: execute_coproc: coproc [12494:dd_sda] still exists
sdb PID: 12496
However this:
for drive in sda sdb
do
coproc_name=dd_${drive}
coproc $coproc_name { dd if=/dev/zero of=/dev/$drive status=progress 2>&1; }
pid_var="${coproc_name}_PID"
echo "$drive PID: ${!pid_var}"
done
doesn't work for the second coprocess:
sda PID: 12759
./wipe.sh: line 39: warning: execute_coproc: coproc [12759:dd_sda] still exists
sdb PID:
When hardcoding the name using if condition, it also works:
for drive in sda sdb
do
coproc_name=dd_${drive}
if [[ "$drive" == 'sda' ]]
then
coproc dd_sda { dd if=/dev/zero of=/dev/$drive status=progress 2>&1; }
elif [[ "$drive" == 'sdb' ]]
then
coproc dd_sdb { dd if=/dev/zero of=/dev/$drive status=progress 2>&1; }
fi
pid_var="${coproc_name}_PID"
echo "$drive PID: ${!pid_var}"
done
sda PID: 12998
./wipe.sh: line 39: warning: execute_coproc: coproc [12998:dd_sda] still exists
sdb PID: 13000
Upvotes: 0
Views: 937
Reputation: 31
Here is my final solution, massively inspired by @Diego Torres Milano (thanks again).
#!/usr/bin/env bash
drives=$@
number_of_drives=$#
drives_to_wipe=($drives)
stripped_drive_path() {
echo $1 | awk -F/ '{ print $3 }'
}
wipe() {
echo "Filling \`$1\` drive with zeros:"
dd if=/dev/zero of=$1 status=progress 2>&1
}
echo "Total number of drives: $number_of_drives"
# Wipe the drives in parallel
iteration=1
for drive in $drives; do
drive_name=$(stripped_drive_path $drive)
coproc_name=dd_${drive_name}
eval coproc $coproc_name "{ wipe $drive; }"
if [[ "$iteration" == 1 ]]; then
echo 'Feel free to ignore the following warnings'
fi
((iteration++))
done
# Display the progress
iteration=1
# Run until all drives are wiped
while [[ ${#drives_to_wipe[@]} > 0 ]]; do
for drive in $drives; do
drive_name=$(stripped_drive_path $drive)
coproc_name=dd_${drive_name}
# Move one line below the "Filling drive with zeros" message
if [[ $iteration > 1 ]]; then
tput cud 1
fi
# Read the drive's current status from the coprocess
if read -r -d $'\r' -u ${!coproc_name[0]} line &> /dev/null; then
tput el # Clear line
echo -e "$line\r\n"
else
# Remove the finished drive from the list
for i in ${!drives_to_wipe[@]}; do
if [ "${drives_to_wipe[$i]}" == "$drive" ]; then
unset drives_to_wipe[$i]
fi
done
tput el # Clear line
echo -e "$(($number_of_drives - ${#drives_to_wipe[@]}))/$number_of_drives done!\r\n"
fi
# Move back one line up
if [[ $iteration > 1 ]]; then
tput cuu 1
fi
done
# Move two lines up for each drive
if [[ ${#drives_to_wipe[@]} > 0 ]]; then
tput cuu $(($number_of_drives * 2))
fi
((iteration++))
done
echo "All drives ($number_of_drives) wiped."
Which results in such output:
$ sudo bash ./script.sh /dev/sda /dev/sdb
Total number of drives: 2
Feel free to ignore the following warnings
./wipe.sh: line 24: warning: execute_coproc: coproc [85994:dd_sda] still exists
Filling `/dev/sda` drive with zeros:
1/2 done!
Filling `/dev/sdb` drive with zeros:
2/2 done!
All drives (2) wiped.
Upvotes: 3
Reputation: 69426
Using coprocesses you can do something like this
#! /bin/bash
coproc DD1 { dd if=/dev/zero of=$drive1 status=progress 2>&1; }
coproc DD2 { dd if=/dev/zero of=$drive2 status=progress 2>&1; }
coproc DD3 { dd if=/dev/zero of=$drive3 status=progress 2>&1; }
el=$(tput el)
cuu1=$(tput cuu1)
IFS=
while :
do
for n in {1..3}
do
v="DD$n"
if read -r -d $'\r' -u ${!v[0]} line
then
printf '%s%s: %s\r\n' "$el" "$v" "$line"
else
printf '%s: Done\r\n' "$el" "$v"
fi
done
for n in {1..3}
do
printf '%s' "$cuu1"
done
done
my dd
does not have status
so I'm assuming dd
adds a \r
instead of \n
.
for loop (unfortunately eval seems to be the only option, don't know why coproc
is not take the name from a variable though)
for n in {1..3}
do
eval coproc "DD${n}" "{ dd if=/dev/zero of=\$drive$n status=progress 2>&1; }"
done
Upvotes: 2
Reputation: 27370
Redirect the status of each dd
to a file, and print those files repeatedly while clearing the old output.
dd status=progress
from GNU coreutils prints its status to stderr, so use 2>
to redirect the status info.
The status is updated by overwriting the current line with \r
followed by the new status. Since \r
only works for single lines, updating four different lines needs terminal control sequences, e.g. ANSI escape codes which can be printed conveniently using commands like clear
and tput
.
outfile_prefix=/tmp/wipe-status-$$-
for drive in "$@"; do
(
echo "Wiping $drive"
# dummy-version for testing ...
dd if=/dev/zero of=/dev/null bs=1 count=5M status=progress 2>&1
# ... if you are happy with the output, replace it with
# dd if=/dev/zero of="$drive" status=progress 2>&1
) > "$outfile_prefix$BASHPID" &
done
clear -x # clear the currently visible screen
while # do-while loop
tput home # move cursor to top left
sed '' "$outfile_prefix"* # print files like `cat`, but with \n at the end
jobs %% &> /dev/null # while jobs are running
do
sleep 1
done
rm "$outfile_prefix"*
Above code is inefficient if you are wiping many big or slow drives, because the status files keep growing (even though they seem to contain only one line) and we keep printing those growing files over and over again.
If you run into problems, try to increase the sleep time. Maybe sed 's/.*\r//'
instead of sed ''
can speed things up. tail -c100
would definitively help, but inserts the temp. file names into the output.
The correct way to handle situation would be ...
mkfifo
command).status=progress
and periodically ...
kill -s USR1
for all* running dd
jobs,* At some point, only a few dd
jobs will run while others already finished and closed their fifos, which complicates that process a bit. This is the main reason I sticked with files for this answer.
Upvotes: 1