CODEWITHSUNDEEP
symfonydoctrinebatch-processing
Jeroen Minnaert
Jeroen Minnaert

Reputation: 645

Doctrine Batch Processing doesn't clear memory?

I am trying to use a technique suggested by Doctrine 2 to process a large number of objects. The technique suggests that by using an iterator and by detaching after processing each iteration, memory usage should be kept to a minimum (they speak about an increase of a few KB for processing 10000 records).

However, when I try to do this, I do not see any objects freed. In fact, I am retrieving a little more than 2000 assets and this increases my memory usage by 90 MB. Clearly, these objects are not freed. Can anyone tell me what I am doing wrong? My code looks as follows:

$profiles = //array of Profile entities 
$qb = $this->createQueryBuilder('a')
            ->addSelect('file')
            ->leftJoin($profileNodeEntityName, 'pn', JOIN::WITH, 'pn.icon = a OR pn.asset = a')
            ->leftJoin(
                $profileEntityName,
                'p',
                JOIN::WITH,
                'pn.profile = p OR p.logo = a OR p.background = a OR p.pricelistAsset = a OR p.pdfTplBlancAsset = a OR p.pdfTplFrontAsset = a OR p.pdfTplBackAsset = a'
            )
            ->innerJoin('a.currentFile', 'file')
            ->where('p IN (:profiles)')
            ->setParameter('profiles', $profiles)
            ->distinct(true);

        $iterableResult = $qb->getQuery()->iterate();

        $start = memory_get_usage() / 1024;
        while (($row = $iterableResult->next()) !== false) {
            // process $row[0]
            $this->getEntityManager()->detach($row[0]);
        }
        $end = memory_get_usage() / 1024 - $start;
        // $end is more of less equal to 90000 or 90 MB

Thanks!

Upvotes: 1

Views: 1209

Answers (1)

csongor.zalatnai
csongor.zalatnai

Reputation: 46

You should also detach the related entities, or set cascade={"detach"} on the associations.

Upvotes: 2

Related Questions