kostiamol
kostiamol

Reputation: 309

Fast file content generating via bash

The task is to write script in bash that will be generating numbers from 1 to some number while the size of file is less than 5 Mb. I've written the script but it is too slow. How can I improve it's speed to 2 seconds?

#!/bin/bash
if ! [ -f task3.txt ]
then
        touch task3.txt
fi
limit=5242880
size=`wc -c < task3.txt`
i=1
while [ $size -le $limit ]
do
        echo "$i " >> task3.txt
        i=$((i+1))
        size=`wc -c < task3.txt`
done

Upvotes: 0

Views: 99

Answers (1)

Wintermute
Wintermute

Reputation: 44043

Instead of running to the file system and counting every byte again every time to figure out how much you've written, keep the count yourself. As it is currently written, you could use

size=$((size + ${#i} + 2))

inside the loop. Here ${#i} is the length of $i, and it's two more per loop because you echo a space and a newline in addition to it. If the space is a typo, as I suspect, then your loop could be

while [ $size -le $limit ]
do
    echo "$i" >> task3.txt     # cut space
    i=$((i+1))
    size=$((size + ${#i} + 1)) # only +1 here.
done

Furthermore, to avoid opening and closing the pipe all the time, you could write

while [ $size -le $limit ]
do
    echo "$i"                  # cut space
    i=$((i+1))
    size=$((size + ${#i} + 1)) # only +1 here.
done >> task3.txt

this doesn't cut the running time down quite as much as removing the wc -c, but it was still ~50% (so worth it).

Addendum for great speed:

Another way to do this that is much, much faster than the manual loop is to abuse seq as follows:

left=$((limit - size))      # bytes left to write

seq 1 $left | head -c $left >> task3.txt

This abuses the fact that head -c $left stops reading after $left bytes, at which point seq's stdout closes and seq also stops writing, and that seq 1 $left will print more than $left bytes of output. The caveat is that it writes not n numbers but n bytes, so the last number in the output may be truncated. It is not in this particular case when starting from an empty file, but that is by chance.

Thanks to @gniourf_gniourf for coming up with most of this one.

Upvotes: 1

Related Questions