CODEWITHSUNDEEP
bashawkoutput
B Team
B Team

Reputation: 101

awk output is acting weird

cat TEXT | awk -v var=$i -v varB=$j '$1~var , $1~varB {print $1}' > PROBLEM HERE

I am passing two variables from an array to parse a very large text file by range. And it works, kind of.

if I use ">" the output to the file will ONLY be the last three lines as verified by cat and a text editor.

if I use ">>" the output to the file will include one complete read of TEXT and then it will divide the second read into the ranges I want.

if I let the output go through to the shell I get the same problem as above.

Question: It appears awk is reading every line and printing it. Then it goes back and selects the ranges from the TEXT file. It does not do this if I use constants in the range pattern search.

I undestand awk must read all lines to find the ranges I request.

  1. why is it printing the entire document?
  2. How can I get it to ONLY print the ranges selected?

This is the last hurdle in a big project and I am beating my head against the table.

Thanks!

Upvotes: 0

Views: 285

Answers (4)

Bright
Bright

Reputation: 23

Here's what happened. I used an array to input into my variables. I set the counter for what I thought was the total length of the array. When the final iteration of the array was reached, there was a null value returned to awk for the variable. This caused it to print EVERYTHING. Once I correctly had a counter with the correct number of array elements the printing oddity ended.

As far as the > vs >> goes, I don't know. It did stop, but I wasn't as careful in documenting it. I think what happened is that I used $1 in the print command to save time, and with each line it printed at the end it erased the whole file and left the last three identical matches. Something to ponder. Thanks Ed for the honest work. And no thank you to Robo responses.

Upvotes: 0

Ed Morton
Ed Morton

Reputation: 203189

It sounds like this is what you want:

awk -v var="foo" -v varB="bar" '$1~var{f=1} f{print $1} $1~varB{f=0}' file

e.g.

$ cat file
1
2
foo
3
4
bar
5
foo
6
bar
7

$ awk -v var="foo" -v varB="bar" '$1~var{f=1} f{print $1} $1~varB{f=0}' file
foo
3
4
bar
foo
6
bar

but without sample input and expected output it's just a guess and this would not address the SHELL behavior you are seeing wrt use of > vs >>.

Upvotes: 1

karakfa
karakfa

Reputation: 67467

Aside from the typo, you can't use variables in //, instead you have to specify with regular ~ match. Also quote your shell variables (here is not needed obviously, but to set an example). For example

seq 1 10 | awk -v b="3" -v e="5" '$0 ~ b, $0 ~ e'

should print 3..5 as expected

Upvotes: 1

Kent
Kent

Reputation: 195029

give this a try, you didn't assign varB in right way:

 yours: awk -v var="$i" -varB="$j" ...
 mine : awk -v var="$i" -v varB="$j" ...
                         ^^

Upvotes: 1

Related Questions