CODEWITHSUNDEEP
unixfile-ioawk
Spyros
Spyros

Reputation: 259

How to match a list of strings in two different files using a loop structure?

I have a file processing task that I need a hand in. I have two files (matched_sequences.list and multiple_hits.list).

INPUT FILE 1 (matched_sequences.list):

>P001 ID
 ABCD .... (very long string of characters)

>P002 ID
 ABCD .... (very long string of characters)

>P003 ID
ABCD ... ( " " " " )

INPUT FILE 2 (multiple_hits.list):

ID1
ID2
ID3
....

What I want to do is match the second column (ID2, ID4, etc.) with a list of IDs stored in multiple_hits.list. Then create a new matched_sequences file similar to the original but which excludes all IDs found in multiple_hits.list (about 60 out of 1000). So far I have:

#!/bin/bash
X=$(cat matched_sequences.list | awk '{print $2}')
Y=$(cat multiple_hits.list | awk '{print $1}')

while read matched_sequenes.list
do
[ $X -ne $Y ] && (cat matched_sequences.list | awk '{print $1" "$2}') > new_matched_sequences.list
done

I get the following error raised: -bash: read: `matched_sequences.list': not a valid identifier Many thanks in advance!

EXPECTED OUTPUT (new_matched_sequences.list):

Same as INPUT FILE 1 with all IDs in multiple_hits.list excluded

Upvotes: 0

Views: 101

Answers (2)

jaybee
jaybee

Reputation: 955

A shorter awk answer is possible, with a tiny script reading first the file with the IDs to exclude, and then the file containing the sequences. The script would be as follows (comments make it long, it's just three useful lines in fact:

BEGIN { grab_flag = 0 }
# grab_flag will be used when we are reading the sequences file
# (not absolutely necessary to set here, though, because we expect the file will start with '>')

FNR == NR { hits[$1] = 1 ; next } # command executed for all lines of the first file: record IDs stored in multiple_hits.list
# otherwise we are reading the second file, containing the sequences:
/^>/ { if (hits[$2] == 1) grab_flag = 0 ; else grab_flag = 1 } # sets the flag indicating whether we have to output the sequence or not
grab_flag == 1 { print }

And if you call this script exclude.awk, you will invoke it this way:

awk -f exclude.awk multiple_hits.list matched_sequences.list

Upvotes: 0

konsolebox
konsolebox

Reputation: 75588

#!/usr/bin/awk -f
function chomp(s) {
    sub(/^[ \t]*/, "", s)
    sub(/[ \t\r]*$/, "", s)
    return s
}
BEGIN {
    file = ARGV[--ARGC]
    while ((getline line < file) > 0) {
        a[chomp(line)]++
    }
    RS = ""
    FS = "\n"
    ORS = "\n\n"
}
{
    id = chomp($1)
    sub(/^.* /, "", id)
}
!(id in a)

Usage:

awk -f script.awk matched_sequences.list multiple_hits.list > new_matched_sequences.list

Upvotes: 1

Related Questions