CODEWITHSUNDEEP
phpmemorysymfony1
SmootQ
SmootQ

Reputation: 2122

Memory usage increasing after every loop

I have a foreach loop, reading a file containing about 200000 lines, this is my PHP : 

foreach ($this->file as $row)
      {
        if ($this->file->valid())
        {

          //init client array
          $this->initInvoicesArray($row);
          $this->prepareInvoice();
          $row = null;
          $this->key = $this->file->key();

          //add msisdn to msisdn array, and client to clients array
          self::$MsisdnArray[] = $this->msisdn;
          self::$InvoicesArray[$this->msisdn] = $this->client;

          if ($i % 3000 == 0) 
          {
            //get valid users from table
            $this->prepareAndSaveValidClients();
          }

          $i++;
        }
      }

and this is the prepareAndSaveValidClients() function :

public function prepareAndSaveValidClients(){
    $query = Doctrine_Query::create()
        ->select('p.gender, p.email2, u.username, u.first_name, u.last_name, u.email_address, u.is_active, p.msisdn, p.user_id, p.city_id, p.street, p.zipcode, p.msisdn_status')
        ->from('sfGuardUser u')
        ->innerJoin('u.Profile as p ON u.id = p.user_id')
        ->whereIn('p.msisdn', self::$MsisdnArray)
        ->whereIn('p.status', self::$AllowedStatus);

    $results = $query->fetchArray();

    //instanciat an object collection for payment_notifications
    $collection = new Doctrine_Collection("payment_notifications");

    if (!empty($results))
    {
      foreach ($results as $key => $client)
      {
        $invoice = self::$InvoicesArray[$client['Profile']['msisdn']];

        $this->initInvoicesArray($invoice);
        $this->prepareInvoice();

        $this->prepareUserProfile($client);
        $this->prepareClient();

        $paymentNotifications = new paymentNotifications();
        $paymentNotifications->fromArray($this->client);

        $collection->add($paymentNotifications);

        $tel = $client['Profile']['msisdn'];
        $client = null;
      }

      $collection->save();

      //clear memory
      $results = null;
      $collection = null;
      self::$MsisdnArray = null;
      self::$InvoicesArray = null;


      $this->logSection('tel num', 'added :' . $tel . ' Memory usage : ' . memory_get_usage());
      $duration = microtime(true) - $this->startTime;
      $this->logSection('payment : ', sprintf('added in %s', $duration));
    }
  }

As for the functions :

   $this->initInvoicesArray($invoice);
   $this->prepareInvoice();

   $this->prepareUserProfile($client);
   $this->prepareClient();

They are just for preparing $this->client

This is Memory Usage as it's displayed for every loop :

>> tel num   added :0699946185 Memory usage : 89287596
>> payment :  added in 8.6373870372772
>> tel num   added :0699983919 Memory usage : 165854544
>> payment :  added in 18.373502969742
>> tel num   added :0699949623 Memory usage : 241338788
>> payment :  added in 29.336947917938
>> tel num   added :0699854750 Memory usage : 319173092
>> payment :  added in 40.880628824234

As you can see, I tried to free memory of these variables :

$results = null;
$collection = null;
self::$MsisdnArray = null;
self::$InvoicesArray = null;

But in vain, Memory Usage keeps increasing after every loop, which results in a An Allowed Memory size exausted fatal error. How can I optimize it?

Thank you.

Upvotes: 6

Views: 1661

Answers (3)

SmootQ
SmootQ

Reputation: 2122

I solved the problem 2 months ago, and the Memory_usage became stable, but I forgot to post the solution here.

What I did, is that I removed Doctrine, and I used mere PDO and Sql for the select query.

Upvotes: 0

lsouza
lsouza

Reputation: 2488

After calling $this->prepareAndSaveValidClients(); you could also use the gc_collect_cycles() function. It will help with possible memory leaks.

Upvotes: 1

Marek
Marek

Reputation: 7433

Doctrine objects have circular references that php has trouble to free. To solve this, doctrine provides free() method, call it on the collection just before setting it to null:

$collection->free(true);
$collection = null;

Upvotes: 3

Related Questions