CODEWITHSUNDEEP
regexbashawk
onur
onur

Reputation: 6365

How can I use bash variable in awk with regexp?

I have a file like this (this is sample):

71.13.55.12|212.152.22.12|71.13.55.12|8.8.8.8
81.23.45.12|212.152.22.12|71.13.55.13|8.8.8.8
61.53.54.62|212.152.22.12|71.13.55.14|8.8.8.8
21.23.51.22|212.152.22.12|71.13.54.12|8.8.8.8
...

I have iplist.txt like this:

71.13.55.
12.33.23.
8.8.
4.2.
...

I need to grep if 3. column starts like in iplist.txt.

Like this:

71.13.55.12|212.152.22.12|71.13.55.12|8.8.8.8
81.23.45.12|212.152.22.12|71.13.55.13|8.8.8.8
61.53.54.62|212.152.22.12|71.13.55.14|8.8.8.8

I tried:

for ip in $(cat iplist.txt); do
    awk -v var="$ip" -F '|' '{if ($3 ~ /^$var/) print $0;}' text.txt
done

But bash variable does not work in /^ / regex block. How can I do that?

Upvotes: 0

Views: 131

Answers (3)

jas
jas

Reputation: 10865

First, you can use a concatenation of strings for the regular expression, it doesn't have to be a regex block. You can say:

'{if ($3 ~ "^" var) print $0;}'

Second, note above that you don't use a $ with variables inside awk. $ is only used to refer to fields by number (as in $3, or $somevar where somevar has a field number as its value).

Third, you can do everything in awk in which case you can avoid the shell loop and don't need the var:

awk -F'|' 'NR==FNR {a["^" $0]; next} { for (i in a) if ($3 ~ i) {print;next} }' iplist.txt r.txt
71.13.55.12|212.152.22.12|71.13.55.12|8.8.8.8
81.23.45.12|212.152.22.12|71.13.55.13|8.8.8.8
61.53.54.62|212.152.22.12|71.13.55.14|8.8.8.8

EDIT
As rightly pointed out in the comments, the .s in the patterns will match any character, not just a literal .. Thus we need to escape them before doing the match:

awk -F'|' 'NR==FNR {gsub(/\./,"\\."); a["^" $0]; next} { for (i in a) if ($3 ~ i) print }' iplist.txt r.txt

I'm assuming that you only want to output a given line once, even if it matches multiple patterns from iplist.txt. If you want to output a line multiple times for multiple matches (as your version would have done), remove the next from {print;next}.

Upvotes: 3

user4401178
user4401178

Reputation:

Here is a way with bash, sed and grep, it's straight forward and I think may be a bit cleaner than awk in this case:
IFS=$(echo -en "\n\b") && for ip in $(sed 's/\./\\&/g' iplist.txt); do grep "^[^|]*|[^|]*|${ip}" r.txt done

Upvotes: 1

muru
muru

Reputation: 4887

Use var directly, instead of in /^$var/ ( adding ^ to the variable first):

awk -v var="^$ip" -F '|' '$3 ~ var' text.txt

By the way, the default action for a true condition is to print the current record, so, {if (test) {print $0}} can often be contracted to just test.

Upvotes: 2

Related Questions