CODEWITHSUNDEEP
php
Vikas
Vikas

Reputation: 55

php Pulling a single line from HUGE text file

I'm ok with PHP but probably not half as good as some of you guys on here.

I am basically trying to find a way to grab a line from a huge and I mean huge text file.... its basically a list of keywords I want to call by line number but without preferably going through them all before I get to that line.....otherwise couldmcrash my server obviously.

At the moment im using this 

$lines = file('http://www.mysite.com/keywords.txt');
foreach ($lines as $line_num => $line) {
   echo "$line_num";
}

This works but im sure theres gotta be a better way of doing to save on usuage because this is putting the whole file into the memory and if I can simply say to php give me line number 97, would umm RULE....

Hope you guys can come up with a solution as your much smarter than me :P ty

Upvotes: 3

Views: 417

Answers (3)

LSerni
LSerni

Reputation: 57453

If the lines are just text and variable in length, you can't know which line is #97; the only thing that makes it 97th is that there are 96 lines before.

So you need to read the whole file up to that point (this is what SplFileObject does):

$fp = fopen("keywords.txt", "r");
while($line--)
{
    if (feof($fp))
        // ERROR: line does not exist
    $text = fgets($fp, 1024); // 1024 = max length of one line
}
fclose($fp);

But if you can store a line number before each line, i.e. the file is

...
95 abbagnale
96 abbatangelo
97 abbatantuono
98 ...

then you can implement a sort of binary search:

- start with s1 = 0 and s2 = file length
- read a keyword and line number at seek position s3 = (s1+s2)/2 (*)
- if line number is less than desired, s1 = s3; else s2 = s3; and repeat previous step.
- if line number is the one desired, strip the number from the text and you get the keyword.

(*) since the line most likely will not start exactly at s#, you need two fgets: one to get rid of the spurious half keyword, the second to read the line number. When you get "close", it will be faster to read a bigger chunk and split it into lines. For example, you seek line 170135 and read in line 170180: what you'd better do is rewind the seek position by one kilobyte, read in a kilobyte of data, and seek 170135 in there.

Or, if the lengths of the various lines are not too different, it could be worthwhile to store a fixed size line (here the "#" should actually be spaces, and in the line length you need to count the line terminator, \n or \r\n):

abbagnale#########
abbatangelo#######
abbatantuono######

and then, say that each keyword is 32 bytes,

$fp = fopen("keywords.txt", "r");
fseek($fp, 97 * 32, SEEK_SET);
$text = trim(fgets($fp, 32));
fclose($fp);

would be more or less instantaneous.

If the file is on a remote server though, you still need to download the Whole file (up to the desired line), and you'd be better served by placing a "scanner" script on the remote server that could run the search. Then you could run

$text = file_get_contents("http://www.mysite.com/keywords.php?line=97");

and get your line in milliseconds.

Upvotes: 2

Manishearth
Manishearth

Reputation: 16198

There isn't any way to get 'line number x' from a file in pretty much any language without having to read it first some way or the other. A line, after all, is just the stuff between two end-of-line characters. Whereas picking up 'character number x' from a file can be done without loading the whole file (with some difficulty), picking up 'line number x' can't be done without loading all lines till x (and in most methods, you need to load all lines)

A method in which you load all the lines till line x is the following (using fgets):

$f = fopen('http://www.mysite.com/keywords.txt');
$i=97
$text=""
while (($text = fgets($f,2048)) !== false && $i>0) {
       $i--
}
echo $text

Upvotes: 0

Anup Singh
Anup Singh

Reputation: 1573

Use SplFileObject

 $file = "test.txt";
  $line_number = 1000;
  $file_obj = new SplFileObject( $file );
    /*** seek to the line number ***/
  $file_obj->seek( $line_number );

   /*** return the current line ***/
   echo  $file_obj->current();

Upvotes: 2

Related Questions