CODEWITHSUNDEEP
bash
van
van

Reputation: 9449

Improve bash script to echo redirect within loop

In bash I have a while do done loop in which I echo and redirect this to a file, i do this about 1 million times so I'm looking to make it faster.

  1. How can i make the code more efficient?
  2. Will mounting the tmp directory in RAM make it faster? mount -t tmpfs -o size=512m tmpfs /tmp
while $count<1000000
do
echo "test statement redirected into file a million times" >>/tmp/output.txt 
done

Upvotes: 2

Views: 193

Answers (1)

John Kugelman
John Kugelman

Reputation: 361585

When the redirection is inside the loop the output file is opened and closed every single iteration. If you redirect the entire loop it's opened for the duration of the loop, which is much, much faster.

for ((i = 0; i < 1000000; ++i)); do
    echo "test statement redirected into file a million times"
done > /tmp/output.txt

(Note also the updated loop logic.)

If you want to make it even faster, you can speed it up by simplifying this to one or two commands that loop internally rather than looping in the shell script. yes will output a fixed string over and over. Combine it with head to control how many lines are printed.

yes 'test statement redirected into file a million times' | head -1000000 > /tmp/output.txt

Will mounting the tmp directory in RAM make it faster?

Maybe, but I wouldn't bother. That's a system-wide change for a local problem. If you don't need the file on disk, it raises the question: do you even need to create the file in the first place? Why do you need a file with the same line repeated a million times? Maybe this is an XY problem. Perhaps you don't need the file at all.

For instance, if you're creating it and immediately passing it to a single subsequent command, you could try using process substitution instead to avoid needing a temp file.

yes 'test statement redirected into file a million times' | head -1000000 > /tmp/output.txt
command /tmp/output.txt

becomes

command <(yes 'test statement redirected into file a million times' | head -1000000)

Upvotes: 4

Related Questions