CODEWITHSUNDEEP
phpfile
Marco Demaio
Marco Demaio

Reputation: 34417

Is this the most efficient way to get and remove first line in file?

I have a script which, each time is called, gets the first line of a file. Each line is known to be exactly of the same length (32 alphanumeric chars) and terminates with "\r\n". After getting the first line, the script removes it.

This is done in this way:

$contents = file_get_contents($file));
$first_line = substr($contents, 0, 32);
file_put_contents($file, substr($contents, 32 + 2)); //+2 because we remove also the \r\n

Obviously it works, but I was wondering whether there is a smarter (or more efficient) way to do this?

In my simple solution I basically read and rewrite the entire file just to take and remove the first line.

Upvotes: 19

Views: 48576

Answers (12)

Oliver
Oliver

Reputation: 865

The solutions here didn't work performantly for me. My solution grabs the last line (not the first line, in my case it was not relevant to get the first or last line) from the file and removes that from that file. This is very quickly even with very large files (>150000000 lines).

function file_pop($file)
{
    if ($fp = @fopen($file, "c+")) {
        if (!flock($fp, LOCK_EX)) {
            fclose($fp);
        }
        $pos = -1;
        $found = 0;
        while ($found < 2) {
            if (fseek($fp, $pos--, SEEK_END) < 0) { // can not seek to position
                rewind($fp); // rewind to the beginnung of the file
                break;
            };
            if (ord(fgetc($fp)) == 10) { // newline
                $found++;
            }
        }
        $lastpos = ftell($fp); // get current position of file
        $lastline = fgets($fp); // get current line

        ftruncate($fp, $lastpos); // truncate file to last position
        flock($fp, LOCK_UN); // unlock
        fclose($fp); // close the file

        return trim($lastline);
    }
}

Upvotes: 0

Richard Dev
Richard Dev

Reputation: 1170

My problem was large files. I just needed to edit, or remove the first line. This was a solution I used. Didn't require to load the complete file in a variable. Currently echos, but you could always save the contents.

$fh = fopen($local_file, 'rb');
echo "add\tfirst\tline\n";  // add your new first line.
fgets($fh); // moves the file pointer to the next line.
echo stream_get_contents($fh); // flushes the remaining file.
fclose($fh);

Upvotes: 1

Kushlendr Pal Ahirwar
Kushlendr Pal Ahirwar

Reputation: 1

I think this is best for any file size 

$myfile = fopen("yourfile.txt", "r") or die("Unable to open file!");
$ch=1;

while(!feof($myfile)) {
  $dataline= fgets($myfile) . "<br>";
  if($ch == 2){
  echo str_replace(' ', '&nbsp;', $dataline)."\n";
  }
  $ch = 2;
} 
fclose($myfile);

Upvotes: 0

Marcos Fernandez Ramos
Marcos Fernandez Ramos

Reputation: 677

This will shift the first line of a file, you dont need to load the entire file in memory like you do using the 'file' function. Maybe for small files is a bit more slow than with 'file' (maybe but i bet is not) but is able to manage largest files without problems.

$firstline = false;
if($handle = fopen($logFile,'c+')){
    if(!flock($handle,LOCK_EX)){fclose($handle);}
    $offset = 0;
    $len = filesize($logFile);
    while(($line = fgets($handle,4096)) !== false){
        if(!$firstline){$firstline = $line;$offset = strlen($firstline);continue;}
        $pos = ftell($handle);
        fseek($handle,$pos-strlen($line)-$offset);
        fputs($handle,$line);
        fseek($handle,$pos);
    }
    fflush($handle);
    ftruncate($handle,($len-$offset));
    flock($handle,LOCK_UN);
    fclose($handle);
}

Upvotes: 6

evalarezo
evalarezo

Reputation: 1153

No need to create a second temporary file, nor put the whole file in memory:

if ($handle = fopen("file", "c+")) {             // open the file in reading and editing mode
    if (flock($handle, LOCK_EX)) {               // lock the file, so no one can read or edit this file 
        while (($line = fgets($handle, 4096)) !== FALSE) { 
            if (!isset($write_position)) {        // move the line to previous position, except the first line
                $write_position = 0;
            } else {
                $read_position = ftell($handle); // get actual line
                fseek($handle, $write_position); // move to previous position
                fputs($handle, $line);           // put actual line in previous position
                fseek($handle, $read_position);  // return to actual position
                $write_position += strlen($line);    // set write position to the next loop
            }
        }
        fflush($handle);                         // write any pending change to file
        ftruncate($handle, $write_position);     // drop the repeated last line
        flock($handle, LOCK_UN);                 // unlock the file
    }
    fclose($handle);
}

Upvotes: 13

Daniel
Daniel

Reputation: 281

I came up with this idea yesterday:

function read_and_delete_first_line($filename) {
  $file = file($filename);
  $output = $file[0];
  unset($file[0]);
  file_put_contents($filename, $file);
  return $output;
}

Upvotes: 28

ghostdog74
ghostdog74

Reputation: 342649

you can iterate the file , instead of putting them all in memory 

$handle = fopen("file", "r");
$first = fgets($handle,2048); #get first line.
$outfile="temp";
$o = fopen($outfile,"w");
while (!feof($handle)) {
    $buffer = fgets($handle,2048);
    fwrite($o,$buffer);
}
fclose($handle);
fclose($o);
rename($outfile,$file);

Upvotes: 5

Tatu Ulmanen
Tatu Ulmanen

Reputation: 124818

Here's one way:

$contents = file($file, FILE_IGNORE_NEW_LINES);
$first_line = array_shift($contents);
file_put_contents($file, implode("\r\n", $contents));

There's countless other ways to do that also, but all the methods would involve separating the first line somehow and saving the rest. You cannot avoid rewriting the whole file. An alternative take:

list($first_line, $contents) = explode("\r\n", file_get_contents($file), 2);
file_put_contents($file, implode("\r\n", $contents));

Upvotes: 2

Frank Farmer
Frank Farmer

Reputation: 39366

I wouldn't usually recommend opening up a shell for this sort of thing, but if you're doing this infrequently on really large files, there's probably something to be said for:

$lines = `wc -l myfile` - 1;
`tail -n $lines myfile > newfile`;

It's simple, and it doesn't involve reading the whole file into memory.

I wouldn't recommend this for small files, or extremely frequent use though. The overhead's too high.

Upvotes: 4

goat
goat

Reputation: 31823

You could store positional info into the file itself. For example, the first 8 bytes of the file could store an integer. This integer is the byte offset of the first real line in the file.

So, you never delete lines anymore. Instead, deleting a line means altering the start position. fseek() to it and then read lines as normal.

The file will grow big eventually. You could periodically clean up the orphaned lines to reduce the file size.

But seriously, just use a database and don't do stuff like this.

Upvotes: 2

Michael D. Irizarry
Michael D. Irizarry

Reputation: 6302

You could use file() method.

Gets the first line

$content = file('myfile.txt');
echo $content[0];

Upvotes: -1

Byron Whitlock
Byron Whitlock

Reputation: 53901

There is no more efficient way to do this other than rewriting the file.

Upvotes: 14

Related Questions