CakePHP gives me Fatal Error for exhausted memory

Question

I get:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 18635837 bytes) in /Users/[...]/cake/libs/cache/file.php on line 135

and I don't understand what could eat out so much RAM.

I have large variables with extensive arrays and data in them. My controller ends like that:

// RENDER
$this->set(compact('var1', 'var2'));
debug(memory_get_usage()); // prints out: 33997240

33MB is nowhere close to 134MB

If I put debug(memory_get_usage()); as the first line of the view, I still get that Fatal Error, which means the problem is not in loops of the view. It doesn't seem to be in the controller either, but rather between the controller and view.

How can I investigate what the problem is and fix the issue?

EDIT< the code of the whole function:

function assignment_results($aid=null, $uid=null){
    if($aid==null){
        $this->Session->setFlash(__('Sorry but my butt got booted. 1907125790'));
        $this->redirect($this->Misc->redirectHome());
    }
    $assignment = $this->EduAssignment->getById($aid);
    // Get User IDS
    if($uid==null){
        $cus = $this->EduCourseUser->getStudentsForCourseId($assignment['EduAssignment']['edu_course_id']);
        foreach ($cus as $cu){
            $uids[]=$cu['EduCourseUser']['user_id'];
        }
    }else{
        $uids[]=$uid;
    }

    // GET WORDING
    $course = $this->EduCourse->getById($assignment['EduCourse']['id']);
    $wt = $this->WritingTranslation->getById($assignment['EduAssignment']['writing_translation_id']);
    $writing['Writing'] = $wt['Writing'];

    if($writing['Writing']['type']== 'song' || $writing['Writing']['type']== 'video')
        $this->paginate['limit'] = 2000;

    $wording = $this->paginate('Word', array('Word.writing_translation_id'=>$wt['WritingTranslation']['id']));
    $word_ids = array();
    foreach($wording as $w){
        $word_ids[]=$w['Word']['id'];
    }

    // CLICKS
    $this->Click->unbindModel(array('belongsTo' => array('Word' )));
    $clicks = $this->Click->getForAssignmentUserIds($assignment['EduAssignment']['id'], $uids);

    // Assign clicks to words
    foreach ($wording as &$wg){
        $num = 0;
        foreach ($clicks as $cl){
            if($wg['Word']['id']==$cl['Click']['word_id']){
                $num++;
            }
        }
        $wg['Word']['click_number'] = $num;
    }

    // List of words by how many times clicked:
    $wording_sorted = $wording;
    // echo(memory_get_usage());
    uasort($wording_sorted, array('TeachController', '_cmp'));
    // debug(memory_get_usage());

    // RENDER
    $this->set(compact('writing', 'wording','wording_sorted', 'assignment', 'course'));
    // debug(memory_get_usage());
}
function _cmp($a, $b){
    return $a['Word']['click_number']<$b['Word']['click_number'];
}

Answer 1

The most likely reason you're getting this error is because you're working with a lot of data at once, and it's never being released when you're done with it. It's basically an optimisation issue.

Given that you have a limit set to 2000 in there, I'm making the assumption you're working with a large database, and that high value might be the single problem. However, I'll highlight some other things that stood out too.

First, note that you're fetching data from the database in a number of places.

$assignment = $this->EduAssignment->getById($aid);

$cus = $this->EduCourseUser->getStudentsForCourseId($assignment['EduAssignment']['edu_course_id']);

$course = $this->EduCourse->getById($assignment['EduCourse']['id']);

$wt = $this->WritingTranslation->getById($assignment['EduAssignment']['writing_translation_id']);

$clicks = $this->Click->getForAssignmentUserIds($assignment['EduAssignment']['id'], $uids);

In between all those queries, you're storing a subset of the queries in new arrays, which leads me to believe your model functions return more data than you actually need. Here's one example in your code:

 foreach ($cus as $cu){
     $uids[]=$cu['EduCourseUser']['user_id'];
 }

Here's another one which seems totally and utterly unnecessary, and it's used totally interchangeably with $wt.

$writing['Writing'] = $wt['Writing'];

Finally, you take $wording, and assign it to $wording_sorted, for sorting. I see you pass both to the view, but do you need both sorted and unsorted forms at exactly the same time? I can't tell you what you should do there, but consider your options.

Here's what you can do to help remedy some of this issues:

Destroy references to data when you're done with it: Unset your query results after you've used them, because they're taking up memory for no reason. Here's a couple:

// after foreach($cus as $cu) { ... }
unset($cus);

// after foreach($wording as $wd) { ... }
unset($clicks);

Of course, you're passing the other things through to your view, so unsetting those would break things.

Remove unnecessary assignments: I've already highlighted one instance where another variable is assigned for no reason. I can't see a reason not to do this:

// $wt = $this->WritingTranslation->getById(...)
$writing = $this->WritingTranslation->getById(...);

// change remaining references to 'wt' to 'writing'

Make sure you're not fetching irrelevant data: It's hard to tell what you're working with, because of the way Cake formats query results, but your models are prime targets for optimisation if the methods within them are returning all the columns in a table, and all its associations, when you only need one or two. Make the conditions you pass to the Cake model more specific if you can.

Use Cake's model count method: You're counting clicks in one place, by fetching them all and using nested iterators. That's a lot of data you don't need when you're just after an integer. Consider creating another model method:

// Click model
function countWordClicks($word_id) {
    return $this->find('count', array('word_id' => $word_id)) ?: 0;
}

// the following thus becomes redundant
$clicks = $this->Click->getForAssignmentUserIds($assignment['EduAssignment']['id'], $uids);

foreach ($wording as &$wg){
    $num = 0;
    foreach ($clicks as $cl){
        if($wg['Word']['id']==$cl['Click']['word_id']){
            $num++;
        }
    }
    $wg['Word']['click_number'] = $num;
}
// unset($clicks);

// and can be replaced with
foreach ($wording as &$wg) {
    $wg['Word']['click_number'] = $this->Click->countWordClicks($wg['Word']['id']);
}

(I can't test that, but it should point you in the right direction.)

That introduces more database queries, but a counter cache field might come in handy to optimise that.

Make your limit more stringent: your limit is set to 2000. I don't know whether that's really small or really high, but if that means you're showing 2000 results per page, simply cutting that down to less than 100 might solve your problem without even having to do anything else.

As said, this is all optimisation stuff, so your code may not be wrong or broken, merely in need of some fine tuning.