CODEWITHSUNDEEP
awk
TilakVarisetty
TilakVarisetty

Reputation: 146

Awk removing the line matching a string

I am writing a script to remove the lines containing the repeated string. For example: 

Epoch Time: 1418027874.795328000 seconds
Data: 4f67675300020000000000000000a6e0d
Epoch Time: 1418027874.807941000 seconds
Data: 4f676753000040caa20641080000a6e
Epoch Time: 1418027874.968753000 seconds
Data: 4f676753000080caa20641080000a6e0d4e40
Epoch Time: 1418027875.131557000 seconds
Epoch Time: 1418027875.131557012 seconds
Data: 4f676753000080caa206410870000a6e0d4e40

I want to remove the instance of another epoch time that repeats twice on line 7.

Upvotes: 0

Views: 116

Answers (2)

gboffi
gboffi

Reputation: 25023

If you had equal epochs in your data file (and you've not) the following would do it nicely

awk 'NF==4{if($3 in e)next;e[$3]} 1' your.data

Testing

% cat ep.dat
Epoch Time: 1418027874.795328000 seconds
Data: 4f67675300020000000000000000a6e0d
Epoch Time: 1418027874.807941000 seconds
Data: 4f676753000040caa20641080000a6e
Epoch Time: 1418027874.968753000 seconds
Data: 4f676753000080caa20641080000a6e0d4e40
Epoch Time: 1418027875.131557000 seconds
Epoch Time: 1418027875.131557000 seconds
Data: 4f676753000080caa206410870000a6e0d4e40
% gawk 'NF==4{if($3 in e)next;e[$3]}1' ep.dat
Epoch Time: 1418027874.795328000 seconds
Data: 4f67675300020000000000000000a6e0d
Epoch Time: 1418027874.807941000 seconds
Data: 4f676753000040caa20641080000a6e
Epoch Time: 1418027874.968753000 seconds
Data: 4f676753000080caa20641080000a6e0d4e40
Epoch Time: 1418027875.131557000 seconds
Data: 4f676753000080caa206410870000a6e0d4e40
% mawk 'NF==4{if($3 in e)next;e[$3]}1' ep.dat
Epoch Time: 1418027874.795328000 seconds
Data: 4f67675300020000000000000000a6e0d
Epoch Time: 1418027874.807941000 seconds
Data: 4f676753000040caa20641080000a6e
Epoch Time: 1418027874.968753000 seconds
Data: 4f676753000080caa20641080000a6e0d4e40
Epoch Time: 1418027875.131557000 seconds
Data: 4f676753000080caa206410870000a6e0d4e40
%

NB I have edited the data file to have two equal epoch times

Upvotes: 0

Gilles Quénot
Gilles Quénot

Reputation: 184975

Is it what you expect ?

$ awk -F'[ .]' '!epochs[$3]++' file

OUTPUT

Epoch Time: 1418027874.795328000 seconds
Data: 4f67675300020000000000000000a6e0d
Epoch Time: 1418027875.131557000 seconds

Upvotes: 1

Related Questions