CODEWITHSUNDEEP
phpunixtext-files
Matt
Matt

Reputation: 22921

Remove first line in text file without allocating memory for entire text file

I have a very large text file and all I need to do is remove one single line from the top of the file. Ideally, it would be done in PHP, but any unix command would work fine. I'm thinking I can just stream through the beginning of the file till I reach \n, but I'm not sure how I do that.

Thanks, Matt Mueller

Upvotes: 4

Views: 2244

Answers (9)

Martin
Martin

Reputation: 1521

Try the following command:

sed -n '2,$p' file

Upvotes: 1

streetparade
streetparade

Reputation: 32908

function cutline($filename,$line_no=-1) {

$strip_return=FALSE;

$data=file($filename);
$pipe=fopen($filename,'w');
$size=count($data);

if($line_no==-1) $skip=$size-1;
else $skip=$line_no-1;

for($line=0;$line<$size;$line++)
if($line!=$skip)
fputs($pipe,$data[$line]);
else
$strip_return=TRUE;

return $strip_return;
}

cutline('foo.txt',1); // deletes line 1 in foo.txt
}

Upvotes: 0

ghostdog74
ghostdog74

Reputation: 342649

you can use a variety of tools in *nix. A comparison of some of the different methods on a file with more than 1.5 million lines. 

$ wc -l < file4
1700589

$ time sed -n '2,$p' file4 > /dev/null

real    0m2.538s
user    0m1.787s
sys     0m0.282s

$ time awk 'NR>1' file4 > /dev/null

real    0m2.174s
user    0m1.706s
sys     0m0.293s

$ time tail -n +2 file4 >/dev/null

real    0m0.264s
user    0m0.067s
sys     0m0.194s

$time  more +2 file4 > /dev/null

real    0m11.771s
user    0m11.131s
sys     0m0.225s

$ time perl -ne 'print if $. > 1' file4 >/dev/null

real    0m3.592s
user    0m3.259s
sys     0m0.321s

Upvotes: 6

yu_sha
yu_sha

Reputation: 4400

tail -n +2 < source > destination

Tail with positive number outputs everything starting with N-th line.

Upvotes: 1

Conrad Meyer
Conrad Meyer

Reputation: 2897

sed -i -e '1d' file will do what you want.

  • -i indicates "in-place"
  • -e means "evaluate this expression"
  • '1d' means, delete the first line

Upvotes: 3

ysth
ysth

Reputation: 98398

Assuming tail from GNU coreutils:

tail -n +2 file > newfile

Upvotes: 1

Daren Thomas
Daren Thomas

Reputation: 70344

I'm a bit rusty on perl, but this might do the trick:

#!/usr/bin/perl
$first = true;
while (<>)
{
    if ($first)
    {
        # skip first line
        $first = false;
    }
    else
    {
        print;
    }
}

and use this script as a filter:

cat myfile.txt | removefirstline.pl > myfile_2.txt

Upvotes: 0

abyx
abyx

Reputation: 72908

I don't know how big is your file, but did you try awk 'NR > 1' {print} ?

Upvotes: 0

Soufiane Hassou
Soufiane Hassou

Reputation: 17750

If your file is flat, you can use sed '1d' file > newfile

Upvotes: 1

Related Questions