Reputation: 35597
I have a file, foo.txt
, containing the following lines:
a
b
c
I want a simple command that results in the contents of foo.txt
being:
a
b
Upvotes: 478
Views: 596037
Reputation: 580
I had several big files more 100GB and sought the most efficient way to do that. Efficiently remove the last two lines of an extremely large text file gives a python scripts. In the same questions, another answer is quite close to the perfect answer. I modified it a bit and get one perfect solution at least I think so. Here is the solution.
truncate --size=-$(tail -n1 myfile | wc -m) myfile
Upvotes: 0
Reputation: 3491
To remove the last line in the zshrc file for example:
cat ~/.zshrc
sed -i '' -e '$ d' ~/.zshrc
cat ~/.zshrc
Upvotes: 2
Reputation: 13937
$
means the last line, and d
means delete:
sed '$d' ~/path/to/your/file/name
Equivalent of the sed -i
:
sed -i '' -e '$ d' ~/path/to/your/file/name
Upvotes: 18
Reputation: 2297
I had trouble with all the answers here because I was working with a HUGE file (~300Gb) and none of the solutions scaled. Here's my solution:
filename="example.txt"
file_size="$(stat --format=%s "$filename")"
trim_count="$(tail -n1 "$filename" | wc -c)"
end_position="$(echo "$file_size - $trim_count" | bc)"
dd if=/dev/null of="$filename" bs=1 seek="$end_position"
Or alternatively, as a one liner:
dd if=/dev/null of=<filename> bs=1 seek=$(echo $(stat --format=%s <filename> ) - $( tail -n1 <filename> | wc -c) | bc )
In words: Find out the length of the file you want to end up with (length of file minus length of length of its last line, using bc
), and set that position to be the end of the file (by dd
ing one byte of /dev/null
onto it).
This is fast because tail
starts reading from the end, and dd
will overwrite the file in place rather than copy (and parse) every line of the file, which is what the other solutions do.
NOTE: This removes the line from the file in place! Make a backup or test on a dummy file before trying it out on your own file!
Upvotes: 164
Reputation: 617
On macOS, head -n -1
wont work but you can use this command:
cat file.txt | tail -r | tail -n +2 | tail -r
tail -r
reverses the order of lines in its input
tail -n +2
prints all the lines starting from the second line in its input
Upvotes: 49
Reputation: 21
awk "NR != `wc -l < text.file`" text.file &> newtext.file
This snippet does the trick.
Upvotes: 0
Reputation: 31
OK processing a good amount of data and the output was OK, but had one junk line.
If I piped the output of the script to:
| sed -i '$ d' I would get the following error and finally no output at all sed: no input files
But | head -n -1 worked!
Upvotes: 3
Reputation: 2600
Here is a solution using sponge (from the moreutils package):
head -n -1 foo.txt | sponge foo.txt
Summary of solutions:
If you want a fast solution for large files, use the efficient tail or dd approach.
If you want something easy to extend/tweak and portable, use the redirect and move approach.
If you want something easy to extend/tweak, the file is not too large, portability (i.e., depending on moreutils
package) is not an issue, and you are a fan of square pants, consider the sponge approach.
A nice benefit of the sponge approach, compared to "redirect and move" approaches, is that sponge preserves file permissions.
Sponge uses considerably more RAM compared to the "redirect and move" approach. This gains a bit of speed (only about 20%), but if you're interested in speed the "efficient tail" and dd approaches are the way to go.
Upvotes: 3
Reputation: 9
You can try this method also : example of removing last n number of lines.
a=0 ; while [ $a -lt 4 ];do sed -i '$ d' output.txt; a=
expr $a + 1
;done
Removing last 4 lines from file(output.txt).
Upvotes: -1
Reputation: 86433
Using GNU sed
:
sed -i '$ d' foo.txt
The -i
option does not exist in GNU sed
versions older than 3.95, so you have to use it as a filter with a temporary file:
cp foo.txt foo.txt.tmp
sed '$ d' foo.txt.tmp > foo.txt
rm -f foo.txt.tmp
Of course, in that case you could also use head -n -1
instead of sed
.
MacOS:
On Mac OS X (as of 10.7.4), the equivalent of the sed -i
command above is
sed -i '' -e '$ d' foo.txt
Upvotes: 604
Reputation: 11
Both of these solutions are here in other forms. I found these a little more practical, clear, and useful:
Using dd:
BADLINESCOUNT=1
ORIGINALFILE=/tmp/whatever
dd if=${ORIGINALFILE} of=${ORIGINALFILE}.tmp status=none bs=1 count=$(printf "$(stat --format=%s ${ORIGINALFILE}) - $(tail -n${BADLINESCOUNT} ${ORIGINALFILE} | wc -c)\n" | bc )
/bin/mv -f ${ORIGINALFILE}.tmp ${ORIGINALFILE}
Using truncate:
BADLINESCOUNT=1
ORIGINALFILE=/tmp/whatever
truncate -s $(printf "$(stat --format=%s ${ORIGINALFILE}) - $(tail -n${BADLINESCOUNT} ${ORIGINALFILE} | wc -c)\n" | bc ) ${ORIGINALFILE}
Upvotes: 1
Reputation: 1319
To remove the last line from a file without reading the whole file or rewriting anything, you can use
tail -n 1 "$file" | wc -c | xargs -I {} truncate "$file" -s -{}
To remove the last line and also print it on stdout ("pop" it), you can combine that command with tee
:
tail -n 1 "$file" | tee >(wc -c | xargs -I {} truncate "$file" -s -{})
These commands can efficiently process a very large file. This is similar to, and inspired by, Yossi's answer, but it avoids using a few extra functions.
If you're going to use these repeatedly and want error handling and some other features, you can use the poptail
command here:
https://github.com/donm/evenmoreutils
Upvotes: 85
Reputation: 1059
Mac Users
if you only want the last line deleted output without changing the file itself do
sed -e '$ d' foo.txt
if you want to delete the last line of the input file itself do
sed -i '' -e '$ d' foo.txt
Upvotes: 33
Reputation: 3917
This is by far the fastest and simplest solution, especially on big files:
head -n -1 foo.txt > temp.txt ; mv temp.txt foo.txt
if You want to delete the top line use this:
tail -n +2 foo.txt
which means output lines starting at line 2.
Do not use sed
for deleting lines from the top or bottom of a file -- it's very very slow if the file is large.
Upvotes: 390
Reputation: 26281
awk 'NR>1{print buf}{buf = $0}'
Essentially, this code says the following:
For each line after the first, print the buffered line
for each line, reset the buffer
The buffer is lagged by one line, hence you end up printing lines 1 to n-1
Upvotes: 14