CODEWITHSUNDEEP
greptcpdumpsystem-administration
Rdang
Rdang

Reputation: 131

How to output tcpdump with grep expression to stdout / file?

I am trying to output the following tcpdump grep expression to a file :

tcpdump -vvvs 1024 -l -A tcp port 80 | grep -E 'X-Forwarded-For:' --line-buffered | awk '{print $2}

I understand it is related to the line-buffered option, that sends the output to stdin. However, if I don't use --line-buffered I don't get any output at all from my tcpdump.

How can I use grep so that it will send my output directly to stdout / file in this case ?

Upvotes: 2

Views: 16130

Answers (1)

user862787
user862787

Reputation:

I am trying to output the following tcpdump grep expression to a file

Then redirect the output of the last command in the pipeline to the file:

tcpdump -vvvs 1024 -l -A tcp port 80 | grep -E 'X-Forwarded-For:' --line-buffered | awk '{print $2}' >file

I understand it is related to the line-buffered option, that sends the output to stdin.

No, that's not with --line-buffered does:

$ man grep

    ...

     --line-buffered
             Force output to be line buffered.  By default, output is line
             buffered when standard output is a terminal and block buffered
             otherwise.

so it doesn't change where the output goes, it just changes when the data is actually written to the output descriptor if it's not a terminal. It's not a terminal in this case - it's a pipe - so, by default, it's block buffered, so if grep writes 4 lines of output, and that's less than a full buffer block (buffer blocks, in this context, are typically 4K bytes in most modern UN*Xes and on Windows, so it's likely that those 4 lines won't fill the buffer), those lines will not immediately be written by grep to the pipe, so they won't show up immediately.

--line-buffered changes that behavior, so that each line is written to the pipe as it's generated, and awk sees it sooner.

You're using -l with tcpdump, which has the same effect, at least on UN*X:

$ man tcpdump

    ...

       -l     Make stdout line buffered.  Useful if you want to see  the  data
              while capturing it.  E.g.,

                     tcpdump -l | tee dat

              or

                     tcpdump -l > dat & tail -f dat

              Note  that on Windows,``line buffered'' means ``unbuffered'', so
              that WinDump will write each character  individually  if  -l  is
              specified.

              -U is similar to -l in its behavior, but it will cause output to
              be ``packet-buffered'', so that the output is written to  stdout
              at  the  end of each packet rather than at the end of each line;
              this is buffered on all platforms, including Windows.

So the pipeline, as you've written it, will cause grep to see each line that tcpdump prints as soon as tcpdump prints it, and cause awk to see each of those lines that contains "X-Forwarded-For:" as soon as grep sees it and matches it.

However, if I don't use --line-buffered I don't get any output at all from my tcpdump.

You'll see it eventually, as long as grep produces a buffer's worth of output; however, that could take a very long time. --line-buffered causes grep to write out each line as it's produced, so it shows up as soon as grep produces it, rather than the buffer is full.

How can I use grep so that it will send my output directly to stdout / file in this case ?

grep is sending its (standard) output to awk, which is presumably what you want; you're extracting the second field from grep's output and printing only that.

So you don't want grep to send its (standard) output directly to the terminal or to a file, you want it to send its output to awk and have awk send its (standard) output there. If you want the output to be printed on your terminal, your command is doing the right thing; if you want it sent to a file, redirect the standard output of awk to that file.

Upvotes: 7

Related Questions