CODEWITHSUNDEEP
linuxshellpipelogfile
user3280180
user3280180

Reputation: 1403

Linux: Pipe output in File until

Does anyone know how I can pipe output to file until filesize reaches limit or is canceled by other condition? Is this possible?

How can I cancel writing output to file ('$cmd >> file.txt 2>&1') with conditions like filesize, time duration ...?

EDIT: I start an application and I need the first ~1000 lines of output, without stopping the application itself! Logging complete application lifetime is not wanted.

Upvotes: 3

Views: 1116

Answers (3)

user184968
user184968

Reputation:

I need the first ~1000 lines of output, without stopping the application itself!

1) I think a simple way to do this is to pipe to sed with two rules. The first rule '1,1000w head-lines.txt' is used to save to the file head-lines.txt, the second rule is used to print.

Write to a file first 1000 lines, print all lines:

$ seq 1000000000000 | sed -n -e '1,1000w head-lines.txt' -e '1,$p'

Write to a file first 1000 lines, print all lines, starting from 1001:

$ seq 1000000000000 | sed -n -e '1,1000w head-lines.txt' -e '1001,$p'

References:

2) A bit more complicated way is to handle SIGPIPE in bash. In bash I will run seq 100000000 which runs long on my server and get only first 1000 lines of its output in file:

$ trap "" SIGPIPE && seq 100000000 2>/dev/null  | head -n 1000 > your.log

When head will get first 1000 line it will end and normally SIGPIPE will be sent to seq. However I call trap in order to ignore SIGPIPE:

$ help trap
trap: trap [-lp] [[arg] signal_spec ...]
    Trap signals and other events.

    If ARG is the null string each SIGNAL_SPEC is ignored by the
    shell and by the commands it invokes.

Upvotes: 1

sashang
sashang

Reputation: 12214

Example:

seq 10000 | tee -i somefile | head -n 1000

This will capture all the data from seq into somefile and 1000 lines will print to stdout. You have to use tee -i to ignore interrupts, otherwise when head ends it will interrupt tee which will interrupt seq.

Upvotes: 4

GHugo
GHugo

Reputation: 2654

For time constraint, use timeout. For size constraint, use tee --max-bytes.

If what you seek is log files management, you should consider using rsyslogd, which handle for you log rotation, size, time, etc.

Upvotes: 1

Related Questions