CODEWITHSUNDEEP
bashpipe
Greg Schoppe
Greg Schoppe

Reputation: 574

clear piped output without interrupting program?

I am running a program (pianobar) piped to a text file, that outputs every second. The resulting file ("pianobarout.txt") needs to be cleared regularly, or it grows to massive proportions. However, I do not want to stop pianobar to clear the file.

I have tried running > pianobarout.txt as well as echo "" > pianobarout.txt, but both cause the system's resources to spike heavily for almost 30 seconds, causing the audio from pianobar to skip. I tried removing the file, but it appears that the file is not recreated after being deleted, and I just lose the pipe.

I'm working from python, so if any library there can help, those are available to me.

Any ideas?

Upvotes: 1

Views: 91

Answers (1)

that other guy
that other guy

Reputation: 123570

If you are currently redirecting with truncation, like yourprogram > file.txt, try redirecting with appending: yourprogram >> file.txt.

There is a big difference between the two when the output file is truncated.

With appending redirection, data is written to the current end of the file. If you truncate it to 0 bytes, the next write will happen at position 0.

With truncating redirection, data is written wherever the last write left off in the file. If you truncate it to 0 bytes, writes will continue at byte 1073741824 where it last left off.

This results in a sparse file if the filesystem supports it (ext2-4 and most Unix fs do), or a long wait while the file is written out if it doesn't (like fat32). A long wait could also be caused by anything following the file, such as tail -f, which has to potentially catch up by reading a GB of zeroes.

Alternatives include yourprogram | split -b 1G - output-, which will write 1GB each to output-aa, output-ab, etc, letting you delete old files at your leasure.

Upvotes: 3

Related Questions