CODEWITHSUNDEEP
shellunixsedawkgrep
nunos
nunos

Reputation: 21389

Using sed to remove lines from a txt file

I have a big text file from which I want to remove some lines that are in another text file. It seems that the sed command in Unix shell is a good way to do this. However, I haven't been able to figure out which flags to use for this. .

database.txt:

this is line 1
this is line 2
this is line 3
this is line 4
this is line 5

lines_to_remove.txt

this is line 1
this is line 3

what_i_want.txt

this is line 2
this is line 4
this is line 5

Upvotes: 0

Views: 302

Answers (3)

Chris Seymour
Chris Seymour

Reputation: 85775

In awk:

$ awk 'NR==FNR{a[$0];next}!($0 in a)' remove.txt database.txt
this is line 2
this is line 4
this is line 5

$ awk 'NR==FNR{a[$0];next}!($0 in a)' remove.txt database.txt > output.txt

Upvotes: 1

Tim Pote
Tim Pote

Reputation: 28029

I would use comm for this:

comm -1 <(sort database.txt) <(sort lines_to_remove.txt) > what_i_want.txt

The command is much better suited to your needs.

NOTE: The <(commmand) syntax is a bashism and is therefore much maligned on SO. It's short hand for the following:

sort database.txt > sorted_database.txt
sort lines_to_remove.txt > sorted_lines_to_remove.txt
comm -1 sorted_database.txt sorted_lines_to_remove.txt > what_i_want.txt

Upvotes: 1

William Pursell
William Pursell

Reputation: 212208

grep is much better suited than sed for this:

grep -Fxv -f lines_to_remove.txt database.txt > what_i_really_really_want.txt

Upvotes: 6

Related Questions