CODEWITHSUNDEEP
shellcsvawkfilterduplicates
TonyW
TonyW

Reputation: 18895

Remove lines that have duplicate entry in the first column

I want to filter my file (tab delimited) by removing the duplicate entries in the first column in each line.

I tried this:

cut -f1 filename.txt | sort | uniq -u > filename_filtered.txt

But this only prints out the first column of the file, is there anyway to filter the first column but print out the entire filtered file?

Upvotes: 1

Views: 1542

Answers (1)

fedorqui
fedorqui

Reputation: 290415

This should make it:

awk '!a[$1]++' file

It keeps track of the fields, printing a line just if its first field haven't appeared yet.

Test

$ cat a
test    hello   bye
test    bye     hello
another thing   here
how     how     how
another blab    bla
text    text    text

$ awk '!a[$1]++' a
test    hello   bye
another thing   here
how     how     how
text    text    text

Upvotes: 2

Related Questions