CODEWITHSUNDEEP
phpgeneratoryield
inf3rno
inf3rno

Reputation: 26137

PHP - Generator, send does not follow the yield order

I want to write example codes step by step how to separate tasks in a generator and move them into 2 or more generators in order to achieve cooperative multitasking between them. You can find all of my tests about this here.

Generators are logical in some way, but I got stucked by a single step, which I cannot explain why it works this way:

The generator:

    $spy = new Object();
    $spy->tasks = array();

    $createGenerator = function ($i1) use ($spy) {

        yield; //(* -> task 1)
        $spy->tasks[] = $i1;
        yield($i1); //(task 1 -> *)

        $i1 = yield; //(* -> task 2)
        //task 2
        $i2 = $i1 + 1;
        $spy->tasks[] = $i2;
        yield($i2); //(task 2 -> *)

        $i2 = yield; //(* -> task 3)
        $i3 = $i2 + 1;
        $spy->tasks[] = $i3;
        yield($i3); //(task 3 -> *)

        $i3 = yield; //(* -> task 4)
        $i4 = $i3 + 1;
        $spy->tasks[] = $i4;
        yield($i4); //(task 4 -> *)

        $i4 = yield; //(* -> task 5)
        $i5 = $i4 + 1;
        $spy->tasks[] = $i5;
        yield($i5); //(task 5 -> *)

    };

The test I waited to succeed, but it failed:

    /** @var Generator $generator */
    $generator = $createGenerator(1);

    $i1 = $generator->send(null);
    $generator->send($i1);
    $i2 = $generator->send(null);
    $generator->send($i2);
    $i3 = $generator->send(null);
    $generator->send($i3);
    $i4 = $generator->send(null);
    $generator->send($i4);
    $i5 = $generator->send(null);

    $this->assertSame($spy->tasks, array(1, 2, 3, 4, 5));
    $this->assertSame(array($i1, $i2, $i3, $i4, $i5), array(1, 2, 3, 4, 5));

The test that unexpectedly succeeds:

    /** @var Generator $generator */
    $generator = $createGenerator(1);

    $i1 = $generator->send(null);
    $generator->send(null); //blank sends needed to skip the yield-yield gaps
    $i2 = $generator->send($i1);
    $generator->send(null);
    $i3 = $generator->send($i2);
    $generator->send(null);
    $i4 = $generator->send($i3);
    $generator->send(null);
    $i5 = $generator->send($i4);

    $this->assertSame($spy->tasks, array(1, 2, 3, 4, 5));
    $this->assertSame(array($i1, $i2, $i3, $i4, $i5), array(1, 2, 3, 4, 5));

Can you explain me this odd behavior of generators by double yield?

Conclusion:

The send() always runs the code from the input of a yield to the output of the next yield. So by running a Generator with send(), it always begins with an input, that's why you cannot get the output of the first yield with a send(), and that's why you always got a null return value by the last send(), before the Generator goes to an invalid state... Sadly the PHP manual has a lack of this information...

Upvotes: 3

Views: 1644

Answers (2)

scenia
scenia

Reputation: 1629

This has confused me for several hours now, but I've managed to find out what exactly is going on and how send works. The important conclusions are:

  • Whether you're sending to (via send) or reading from (via foreach or current()) the Generator first, it will run to the first yield to get "initialised" and then perform your send/receive at that yield. If your first operation is calling next(), it will run all the way to the second yield.
  • Every additional yield will be both the exit and entry point for the next read/send operation.
  • And finally: send() performs both a send and a read operation in that order, regardless of what you do with the result. This means when you call send(), the Generator will move to the next yield while handling your call.

With this in mind, let's walk through your test:

  1. $createGenerator(1) just makes the generator and doesn't run anything, as expected.
  2. $i1 = $generator->send(null); performs a send first, then a read:
    • The Generator runs until it finds the first yield.
    • The null you sent is this yield's value if you were to assign it.
    • The Generator continues running to the next yield, adding $i1 to your task list and then handing execution back, with the yielded value 1.
    • In your test code, this yielded value is assigned to $i1.
  3. $generator->send($i1); now sends this value back into the Generator, again performing a send, then advancing the Generator to the next yield, and reading the yielded value:
    • We're still at the yield($i1);, which previously provided the value 1. This was the exit point and now becomes the entry point receiving the sent value (second conclusion above).
    • However, you're not assigning the value, so the Generator continues to the next yield.
    • The next yield doesn't provide a value, so null is yielded.
    • This isn't assigned in your test code anyway, so everything is fine from the outside view, but inside the Generator, the sent 1 was discarded rather than stored.
  4. $i2 = $generator->send(null); now sends in a null, which again results in a send and read operation, in that order, advancing the Generator in the process:
    • $i1 = yield; //(* -> task 2) is the current yield, which was previously the exit point and now becomes the entry point. Since we just sent in null, that's what is stored in $i1.
    • The Generator now advances, storing null + 1 (so 1) in $i2, storing that in your task list and then yielding it.
    • This yielded 1 is now stored in your test code's $i2. At this point, the previous mistake has leaked out into the test code.
  5. This continues on for the rest of the test, so every step will send in the 1, which will be discarded by the yield statement that sent it out, then the next yield will store the null you send in next and repeat everything. Your resulting array will be [1, 1, 1, 1, 1].

Your working test works because it moves the sent value one step further, to the yield that actually stores what you send. @GhostGambler's Generator works for the opposite reason, it moves the assignment inside your Generator one step closer, to the yield that actually receives the value from your test code.

To illustrate these inner workings, consider the following adapted example:

$createGenerator = function ($i) {

    $in1 = yield("out".$i++); //(* -> task 1)
    echo "in1: $in1";
    $in2 = yield("out".$i++); //(task 1 -> *)
    echo "in2: $in2";
    $in3 = yield("out".$i++); //(* -> task 2)
    echo "in3: $in3";
    $in4 = yield("out".$i++); //(task 2 -> *)
    echo "in4: $in4";
    echo "\nGenerator done with i=$i";
};

$generator = $createGenerator(1);

$out = $generator->send("in1");
echo "\nout1: ";var_dump($out);
$out = $generator->send("in2");
echo "\nout2: ";var_dump($out);
$out = $generator->send("in3");
echo "\nout3: ";var_dump($out);
$out = $generator->send("in4");
echo "\nout4: ";var_dump($out);

Which creates the following output:

in1: in1
out1: string(4) "out2"
in2: in2
out2: string(4) "out3"
in3: in3
out3: string(4) "out4"
in4: in4
Generator done with i=5
out4: NULL

Notice that "out1" (the first yielded value) is never captured because the first operation on the Generator is a send. Consequently, the last read operation fails because the Generator only yields 4 values and we're trying to access the fifth yielded value (since we implicitly discarded the first).

One last gotcha is next(), which is implicitly called at the end of each iteration of a foreach loop. That one is essentially identical to send(null): It sends nothing into the Generator, makes it run to the next yield, and discards the next yielded value (although in a foreach loop, current() is then also called to pick that discarded value up and store it inside the as variable). This makes it rather tricky to use send() inside a foreach.

Upvotes: 2

Ulrich Thomas Gabor
Ulrich Thomas Gabor

Reputation: 6654

Working example

Working example of a generator for your test:

$spy = new stdClass();
$spy->tasks = array();

$createGenerator = function ($i1) use ($spy) {
    yield;
    $spy->tasks[] = $i1;
    $i1 = (yield $i1);

    yield;
    $i2 = $i1 + 1;
    $spy->tasks[] = $i2;
    $i2 = (yield $i2);

    yield;
    $i3 = $i2 + 1;
    $spy->tasks[] = $i3;
    $i3 = (yield $i3);

    yield;
    $i4 = $i3 + 1;
    $spy->tasks[] = $i4;
    $i4 = (yield $i4);

    yield;
    $i5 = $i4 + 1;
    $spy->tasks[] = $i5;
    (yield $i5);
};

Your test:

$generator = $createGenerator(1);

$i1 = $generator->send(null);
$generator->send($i1);
$i2 = $generator->send(null);
$generator->send($i2);
$i3 = $generator->send(null);
$generator->send($i3);
$i4 = $generator->send(null);
$generator->send($i4);
$i5 = $generator->send(null);

print_r($spy);
for ($i = 1; $i <= 5; ++$i) {
    echo ${'i'.$i} . "\n";
}

This gives the desired result:

stdClass Object
(
    [tasks] => Array
        (
            [0] => 1
            [1] => 2
            [2] => 3
            [3] => 4
            [4] => 5
        )

)
1
2
3
4
5

Further hints

See the manual on the send method for further information, which sums up all there is to say about how send works:

Sends the given value to the generator as the result of the current yield expression and resumes execution of the generator.

If the generator is not at a yield expression when this method is called, it will first be let to advance to the first yield expression before sending the value.

You should already know what yield does.

To fully understand the interaction between the generator and your test, it should help you to write down (with a pen on a piece of paper) each step in the execution flow of your source code.

Little note concerning syntax

Also notice the caution box on yield in the manual:

Caution

If you use yield in an expression context (for example, on the right hand side of an assignment), you must surround the yield statement with parentheses. For example, this is valid:

$data = (yield $value);

But this is not, and will result in a parse error:

$data = yield $value;

This syntax may be used in conjunction with the Generator::send() method.

Upvotes: 2

Related Questions