CODEWITHSUNDEEP
bashmatchlines
Marcos Pousada
Marcos Pousada

Reputation: 169

Bash match lines from two files

i have two files:

File_1.txt

PROCESS1 1234
PROCESS2 1235
PROCESS3 1236
PROCESS4 1237

File_2.txt

1.2.3.4 1234
1.2.3.5 1235
1.2.3.6 1236
1.2.3.7 1234
1.2.3.8 1234
1.2.3.9 1235
1.2.3.8 1234
1.2.3.8 1234

The first file, has a process name and it's pid, and the second file, has an IP and a PID too.. what i need is to find the matching PID from the first file in the second file, and output somenthing like this

Desired_output.txt

PROCESS1 1234 1.2.3.4 1
PROCESS1 1234 1.2.3.8 3
PROCESS2 1235 1.2.3.5 1
PROCESS2 1235 1.2.3.9 1
PROCESS3 1236 1.2.3.6 1
PROCESS4 1237  - 0

Where, the first column is the process name, the second column is the PID , the third column are the ips in the file and the fourth column is the times that the ip is listed in the file_2.txt

Thanks in advance

Upvotes: 1

Views: 166

Answers (2)

Idriss Neumann
Idriss Neumann

Reputation: 3838

Using join (who is probably the best solution) :

$ join -j2 <(sort -k2 test1.txt) <(sort -k2 test2.txt)
1234 PROCESS1 1.2.3.4
1234 PROCESS1 1.2.3.7
1234 PROCESS1 1.2.3.8
1234 PROCESS1 1.2.3.8
1234 PROCESS1 1.2.3.8
1235 PROCESS2 1.2.3.5
1235 PROCESS2 1.2.3.9
1236 PROCESS3 1.2.3.6

Then you could try something like :

$ cat test.sh
#!/bin/bash

file1=test1.txt
file2=test2.txt

joinOutput=$(join -j2 <(sort -k2 "$file1") <(sort -k2 "$file2"))
countOutput=$(echo "$joinOutput"|awk '{key=$2" "$1" "$3; if(key in t){t[key]=t[key]+1} else {t[key]=1}} END {for (l in t){print l" "t[l]}}'|sort)
echo "$countOutput"
cat "$file1" <(echo "$countOutput")|cut -d " " -f2|sort|uniq -u|while read; do
    echo "$REPLY - 0"
done
$ ./test.sh
PROCESS1 1234 1.2.3.4 1
PROCESS1 1234 1.2.3.7 1
PROCESS1 1234 1.2.3.8 3
PROCESS2 1235 1.2.3.5 1
PROCESS2 1235 1.2.3.9 1
PROCESS3 1236 1.2.3.6 1
1237 - 0

Otherwise, you could try something like :

$ cat test.sh 
#!/bin/bash

file1=test1.txt
file2=test2.txt

joinOutput=$(
    while read c11 c12; do
        while read c21 c22; do
            [[ $c22 = $c12 ]] && echo "$c11 $c12 $c21"
        done < "$file2"
    done < "$file1"
)

countOutput=$(echo "$joinOutput"|awk '{if($0 in t){t[$0]=t[$0]+1} else {t[$0]=1}} END {for (l in t){print l" "t[l]}}'|sort)
echo "$countOutput"
cat "$file1" <(echo "$countOutput")|cut -d " " -f2|sort|uniq -u|while read; do
    echo "$REPLY - 0"
done
$ ./test.sh
PROCESS1 1234 1.2.3.4 1
PROCESS1 1234 1.2.3.7 1
PROCESS1 1234 1.2.3.8 3
PROCESS2 1235 1.2.3.5 1
PROCESS2 1235 1.2.3.9 1
PROCESS3 1236 1.2.3.6 1
1237 - 0

Upvotes: 2

DWilches
DWilches

Reputation: 23015

Try the join command, it let's you do exactly that.

Only restriction I can think of, is that the files must be sorted. So if you don't care about preserving the order of the output lines you can use join.

Manual page: http://linux.die.net/man/1/join

Upvotes: 2

Related Questions