CODEWITHSUNDEEP
csv
mousey
mousey

Reputation: 11901

Finding unique Values from a file

I have a 6 mb sized csv file. I want to filter the data by column A & Column C so that I need to remove any duplicates. What is the easiest way to do it and how to do it. Any help is very much appreciated.

Upvotes: 3

Views: 7101

Answers (2)

shweta
shweta

Reputation: 101

cat foo.csv | cut -f2 -d , | sort | uniq

It will give you unique ids from 2nd column

cat foo.csv | cut -f1 -d , | sort | uniq

It will give you unique ids from 1st column

-f < number > : column number

-d  < space >< delimiter > : file delimiter

Upvotes: 10

Navi
Navi

Reputation: 8736

Use cut or awk to select fields. Sort and uniq to remove duplicates. FOr example

awk -F"," '{print $1}' A.csv|sort|uniq

Upvotes: 6

Related Questions