AppendIterator doesn't play well with empty generators, how to work around it?

Question

Sometimes, when I have to concatenate multiple iterators into a single one, it is just a single empty iterator. I use AppendIterator for concatenating iterators. When I add an empty Generator from a yielding method to an AppendIterator, I would expect it to become an iterator with 0 elements:

<?php

namespace Tests\cases\domain;

use AppendIterator;
use PHPUnit\Framework\TestCase;

class IteratorTest extends TestCase
{

    public function test_iterator() {
        $fruits = new Fruits(['banana', 'banana', 'orange']);
        $concat = new AppendIterator();
        $concat->append($fruits->getApples());
        $array = iterator_to_array($concat);
        $this->assertEquals(0, count($array));
    }

}

class Fruits
{

    public function __construct(protected array $items) {

    }

    public function getApples(): \Iterator {
        foreach ($this->items as $item) {
            if ($item === 'apple') {
                yield $item;
            }
        }
    }
}

However, when traversing such iterator, it produces an error:

PHPUnit 11.3.6 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.12
Configuration: /home/me/Projects/personal/myproject/framework/phpunit.xml


Exception: Cannot traverse an already closed generator
/home/me/Projects/personal/myproject/tests/cases/domain/IteratorTest.php:15

Time: 00:00.002, Memory: 10.00 MB

There was 1 error:

1) Tests\cases\domain\IteratorTest::test_iterator
Exception: Cannot traverse an already closed generator

/home/me/Projects/personal/myproject/tests/cases/domain/IteratorTest.php:15

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.
Process finished with exit code 2

It seems to be trying to traverse the generator twice, which is unacceptable because generators don't auto rewind.

How do I work around this? I don't see anything wrong with my generator. Everything points to something fishy in the AppendIterator implementation, is this a bug in PHP standard library? Can I work around this while still using generators in my code?

Answer 1

Looks like this is a well-known ancient bug:

https://bugs.php.net/bug.php?id=72692

The workaround is to wrap the empty generator in a NoRewindIterator:

$concat = new \AppendIterator();
$concat->append(new \NoRewindIterator($fruits->getApples()));

Answer 2

I understand your problem! The "Cannot traverse an already closed generator" exception occurs because generators in PHP are single-pass, meaning they cannot be traversed more than once. The AppendIterator is trying to traverse the already closed generator. Here is a solution that keeps using generators while avoiding the problem of closed generators:

We can encapsulate the generator logic in a separate class and make sure the traversal logic is handled correctly:

<?php

namespace Tests\cases\domain;

use AppendIterator;
use IteratorAggregate;
use ArrayIterator;
use PHPUnit\Framework\TestCase;

class IteratorTest extends TestCase
{
    public function test_iterator() {
        $fruits = new Fruits(['banana', 'banana', 'orange']);
        $concat = new AppendIterator();
        $concat->append($fruits->getApples());
        $array = iterator_to_array($concat);
        $this->assertEquals(0, count($array));
    }
}

class Fruits implements IteratorAggregate
{
    protected array $items;

    public function __construct(array $items) {
        $this->items = $items;
    }

    public function getApples(): \Iterator {
        return new ArrayIterator(array_filter($this->items, function ($item) {
            return $item === 'apple';
        }));
    }

    public function getIterator(): \Iterator {
        return $this->getApples();
    }
}

In this solution:

I've used IteratorAggregate and ArrayIterator to handle the generator logic.

The getApples method returns a filtered ArrayIterator instead of a generator.

ArrayIterator is an iterator that can be traversed multiple times without being closed.

With this modification, you should avoid the "Cannot traverse an already closed generator" error and keep your code working as expected.

Hope this is helpful! 🚀