CODEWITHSUNDEEP
php
Rynardt
Rynardt

Reputation: 5557

CSV export script running out of memory

I have a PHP script that exports a MySQL database to CSV format. The strange CSV format is a requirement for a 3rd party software application. It worked initially, but I am now running out of memory at line 743. My hosting service has a limit of 66MB.

My complete code is listed here. http://pastebin.com/fi049z4n

Are there alternatives to using array_splice? Can anyone please assist me in reducing memory usage. What functions can be changed? How do I free up memory?

Upvotes: 3

Views: 4198

Answers (2)

arkascha
arkascha

Reputation: 42925

You have to change your CSV creation strategy. Instead if reading the whole result from the database into an array in memory you must work sequencially:

  • Read a row from the database
  • Write that row into the CSV file (or maybe buffer)
  • Free the row from memory
  • Do this in a loop...

This way the memory consumption of your script does not rise proportionally with the number of rows in the result, but stays (more or less) constant.

This might serve as an untested example to sketch the idea: 

while (FALSE!==($row=mysql_fetch_result($res))) {
  fwrite ($csv_file."\n", implode(',', $row));
}

Upvotes: 7

Vinod Kumar
Vinod Kumar

Reputation: 1489

Increase the memory limit and maximum execution time

For example :

// Change the values to suit you
ini_set("memory_limit","15000M");
ini_set("max-execution_time", "5000");

Upvotes: 0

Related Questions