CODEWITHSUNDEEP
phpcsv
Matt
Matt

Reputation: 95

Reading multiple columns from large CSV files in PHP

I need to read two columns from a large CSV file. The CSV has multiple columns and can sometimes have following properties:

  1. ~25,000 lines
  2. Contain spaces and blank rows
  3. Be uneven (some columns longer than others)

enter image description here

In the example CSV file above, I would be only interested in the codes in the "Buy" and "Sell" columns (columns A and D).

I have written the following code (warning: it's not very elegant) to iterate over all rows and read only the columns I require. I create strings as inputs for 1 large MYSQL query (as opposed to running many small queries).

<?php 
//Increase the allowed execution time 
set_time_limit(0);
ini_set('memory_limit','256M');
ini_set('max_execution_time', 0);     

//Set to detect the ending of CSV files
ini_set('auto_detect_line_endings', true);

$file = "test.csv";

$buy = $sold = ""; //Initialize empty strings

if (($handle = @fopen($file, "r")) !== FALSE) {

while (($pieces = fgetcsv($handle, 100, ",")) !== FALSE) {       

if ( ! empty($pieces[0]) ) {
    $buy .= $pieces[0] ." ";
} 

if ( ! empty($pieces[3]) ) {
   $sold .= $pieces[3] ." ";
} 
}

echo "Buy ". $buy ."<br>"; //Do something with strings...
echo "Sold ". $sold ."<br>";

//Close the file
fclose($handle);  
}

?>

My question is: is this the best way to perform such a task? The code works for smaller test files, but are there short comings I've overlooked in iterating over the CSV file like this?

Upvotes: 3

Views: 1577

Answers (1)

Momin
Momin

Reputation: 868

First, reading any large files is memory consuming if you store them in variables. You may check out reading large files(more than 4GB in unix)

Secondly, you can output the $buy & $sold on the while loop which might be more memory efficient in the way that those two variables are not saved on the memory.

Lastly, Use file seek method in php fseek documentation

Upvotes: 1

Related Questions