CODEWITHSUNDEEP
php
user2599674
user2599674

Reputation: 13

Why do I keep getting memory allocation errors when running my php script?

I'm iterating through a handful of csv files (none that exceed more than 320MB) to clean up and format the files how I need them using the follow script:

while($row = mysqli_fetch_array($rsSelect)){        

    $fileName = $row["file_name"];

    if(strpos($fileName,'DATA') == true){

        $file = $dir.$row["manifest_files"]."";
        echobr($file);
        $file1 = file_get_contents($file, null);

        unset($file);
        $file2 = str_replace('","','    ', $file1);

        unset($file1);
        $file3 = str_replace('"','', $file2);

        file_put_contents($dir.$row["file_name"].".txt",$file3,FILE_USE_INCLUDE_PATH);
        unset($file2);
        unset($file3);

}

I receive the following error no matter how high I set my memory limits within php.ini or unsetting variabeles where I can to free up memory. I've tried setting the limit inside the script as well and still no go. My machine has no shortage of RAM, enough to easily store when manipulating the files as I need them.

ERROR: ( ! ) Fatal error: Out of memory (allocated 683147264) (tried to allocate 364322204 bytes)

Upvotes: 0

Views: 73

Answers (1)

Alex Howansky
Alex Howansky

Reputation: 53591

Using file_get_contents() requires the entire file to be read into memory in one big chunk. You're probably better off using fopen() with a loop around fgets(). This will read just one line at a time:

$fp = fopen('file.csv', 'r');
while (($line = fgets($fp)) !== false) {
    // $line is the whole line
}

Alternatively, you might be able to use fgetcsv() and process individual fields as needed:

$fp = fopen('file.csv', 'r');
while (($row = fgetcsv($fp)) !== false) {
    // $row is an array of values from the line
}

Upvotes: 3

Related Questions