CODEWITHSUNDEEP
linuxbashshellpipelinenetcat
Brian Chrisman
Brian Chrisman

Reputation: 3694

listening on netcat works but not grep'able (or several other utilities)

I'm testing some netcat udp shell tools and was trying to take output and send it through standard pipe stuff. In this example, I have a netcat client which sends 'foo' then 'bar' then 'foo' with newlines for each attempt reading from the listener:

[root@localhost ~ 05:40:20 ]# exec 5< <(nc -d -w 0 -6 -l -k -u 9801)
[root@localhost ~ 05:40:25 ]# awk '{print}' <&5
foo
bar
foo
^C
[root@localhost ~ 05:40:48 ]# awk '{print}' <&5 | grep foo
^C
[root@localhost ~ 05:41:12 ]# awk '{print}' <&5 | grep --line-buffered foo
^C
[root@localhost ~ 05:41:37 ]#
[root@localhost ~ 05:44:38 ]# grep foo <&5
foo
foo
^C
[root@localhost ~ 05:44:57 ]#

I've checked the --line-buffered... and I also get the same behavior from 'od -bc', ie, nothing at all. Grep works on the fd... but if I pipe that grep to anything (like od -bc, cut), I get the same (nothing). Tried prefixing the last grep with stdbuf -oL to no avail. Could > to file and then tail -f it, but feel like I'm missing something.

Update: Appears to be something descriptor/order/timing related. I created a file 'tailSource' and used this instead, which produced the same issue (no output) when I ran echo -e "foo\nfoo\nbar\nfoo" >> tailSource

[root@localhost shm 07:16:18 ]# exec 5< <(tail -n 0 -f tailSource)
[root@localhost shm 07:16:32 ]# awk '{print}' <&5 | grep foo

... and when I run without the '| grep foo', I get the output I'd expect. (GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu))

Upvotes: 1

Views: 557

Answers (2)

geirha
geirha

Reputation: 6171

awk is buffering when its output is not going to a terminal. If you have GNU awk, you can use its fflush() function to flush after every print

gawk '{print; fflush()}' <&5 | grep foo

In this particular case though, you don't need awk and grep, either will do.

awk /foo/ <&5
grep foo <&5

See BashFAQ 9 for more on buffering and how to work around it.

Upvotes: 2

martin
martin

Reputation: 3249

For what it's worth

$ exec 5< <(echo -e "foo\nbar\nfoo")
$ awk '{print}' <&5 | grep foo

prints

foo
foo

where as

$ exec 5< <(echo -e "foo\nbar\nfoo") 
$ awk '{print}' <&5 
foo 
bar 
foo 
$ awk '{print}' <&5
$

the second call does not output anything.

I think this is because the file descriptor is at its end and you need to rewind it, see also this answer to this question question.

In order to use it, you could alternatively either capture it in a temporary file or perform the transformations in one line.

Upvotes: 2

Related Questions