CODEWITHSUNDEEP
bashsortinguniq
Andrew Kennen
Andrew Kennen

Reputation: 163

How to sort,uniq and display line that appear more than X times

I have a file like this:

80.13.178.2
80.13.178.2
80.13.178.2
80.13.178.2
80.13.178.1
80.13.178.3
80.13.178.3
80.13.178.3
80.13.178.4
80.13.178.4
80.13.178.7

I need to display unique entries for repeated line (similar to uniq -d) but only entries that occur more than just twice (twice being an example so flexibility to define the lower limit.)

Output for this example should be like this when looking for entries with three or more occurrences: 

80.13.178.2
80.13.178.3

Upvotes: 16

Views: 12057

Answers (2)

hek2mgl
hek2mgl

Reputation: 157967

With pure awk:

awk '{a[$0]++}END{for(i in a){if(a[i] > 2){print i}}}' a.txt

It iterates over the file and counts the occurances of every IP. At the end of the file it outputs every IP which occurs more than 2 times.

Upvotes: 11

iruvar
iruvar

Reputation: 23364

Feed the output from uniq -cd to awk

sort test.file | uniq -cd | awk -v limit=2 '$1 > limit{print $2}'

Upvotes: 24

Related Questions