Improve documentation about await-generator HOT 9 CLOSED

If you are familiar with JavaScript, yield is just analogous to await. Otherwise, let me assume that you know nothing about async programming in other languages. Furthermore, I assume that you are familiar with PocketMine's scheuler system. I am first going to explain why generators are even related to async, then I will explain how await-generator utilizes this relation to construct an async framework

First, let's look at this simple generator function:

function gen() {
    for($i = 0; $i < 10; $i++){
        $alphabet = yield $i;
        echo "Sent back $alphabet\n";
    }
}

If you read the PHP documentation on generators carefully, you would see that the generator interface allows us to make Generator work like a question-and-answer interface, such that yielding a value is like asking the user a question (what is the $ith alphabet?):

$gen = gen();
while(($i = $gen->current()) !== null) {
    echo "Received $i\n";
    $alphabet = chr(ord('A') + $i);
    $gen->send($alphabet);
}

Test run: https://3v4l.org/2Rg42

$gen->send() sends the parameter global $alphabet the evaluated value of the expression yield $i, and continues the execution of the generator until the next time yield is evaluated.

In fact, we don't have to run $gen->send() immediately. For example, we can run this:

// let's assume the yielded value is the number of milliseconds to sleep
function gen2(){
  $startTime = microtime(true);
  yield 1;
  echo "Time: ", microtime(true) - $startTime;
  yield 2;
  echo "Time: ", microtime(true) - $startTime;
  yield 3;
  echo "Time: ", microtime(true) - $startTime;
  yield 2;
  echo "Time: ", microtime(true) - $startTime;
}

$gen = gen2();
for($tick = 0; $tick < 4; $tick++) {
    $sleepTime = $gen->send();
    usleep($sleepTime * 1000);
}

Test run: https://3v4l.org/njfXL
(Results are slightly inaccurate due to function execution time overhead)

This is nothing interesting, since yield just looks like the same as sleep here. But what if we have multiple generators?

function gen($name, ...$yields){
    foreach($yields as $k => $yield){
        $tick = yield $yield;
        echo "$name looped $k times at tick $tick\n";
    }
}

$gen1 = gen('gen1', 2, 2, 2, 2);
$gen2 = gen('gen2', 1, 1, 1, 1);
$gen3 = gen('gen3', 1, 2, 1, 2);

$remaining1 = 1;
$remaining2 = 1;
$remaining3 = 1;

for($tick = 0; $tick < 10; $tick++) {
    if(--$remaining1 === 0) {
        $remaining1 = $gen1->send($tick);
    }
    if(--$remaining2 === 0) {
        $remaining2 = $gen2->send($tick);
    }
    if(--$remaining3 === 0) {
        $remaining3 = $gen3->send($tick);
    }
}

So while a generator is yielding, we can do other stuff as well! Isn't this similar to callback-style async functions?

You might note that this is already very similar to PocketMine's scheduler system, where each $gen* approximately represents a scheduled task.
(I expanded general loops into explicit 1,2,3 to make code simpler to read)

When a generator completes, we can call $gen->getReturn() to obtain the return value. This is just like a final oneshot channel (in contrast to the Q&A channel through yield) to pass a value from the generator function to the generator holder.

So to conclude the features of generators:

• As you already know, yield can "pause" the function.
• The code holding the Generator object (I call this "generator holder") and the code inside the generator function can communicate with each other. In particular, if the generator yields a value (I call this a "generator query"), it is analogous to calling a procedure in the generator holder. In the first example, this procedure is $i => chr(ord('A') + $i). In the second example, this procedure is calling sleep.
  • Or with more understandable terms, this is analogous to calling a function in the generator holder. But the stack trace happens oppositely: Instead of pushing the stack to run a procedure, we actually pop the stack back to the generator holder. This could cause confusion if you are debugging over an exception trace, but otherwise this makes no semantic difference.
• When the generator holder receives a generator query, it might choose to send back the response later and do other things first. This is analogous to what a callback-style async function does, but in the opposite direction. Running $gen->send() is the callback in this case.
  • In fact, it is literally what happens when you do yield Await::ONCE; in await-generator: https://github.com/SOF3/await-generator/blob/master/await-generator/src/SOFe/AwaitGenerator/Await.php#L328-L330
• The generator can return a value eventually, which is an alternative one-way channel to send data.

So you can easily guess why await-generator uses generators to achieve async.
Users pass a generator into await-generator via Await::g2c. Then await-generator is the generator holder, so it controls how the yielded values are handled and what get sent back to the generator.

Now we come to the details of await-generator. These types of values can be yielded:

• Await::RESOLVE, Await::RESOLVE_MULTI, Await::REJECT: These will evaluate into a callback immediately. This is like the first example, but we return a callback sensitive to the state of your current generator.
• Await::ONCE: This will evaluate when the last Await::RESOLVE returned callback gets called. This is like the third example, except it depends on the previous yield (unlike the third example above, which just depends on the state of $tick).
• Await::ALL and Await::RACE: These are just more complicated versions of Await::ONCE that work over multiple resolves.
• A generator object: This is probably where you get confused. In fact, you may assume that yielding a generator is syntactic sugar for calling Await::g2c again internally.
  • In fact, this is what await-generator does internally: https://github.com/SOF3/await-generator/blob/master/await-generator/src/SOFe/AwaitGenerator/Await.php#L340
  • It involves some additional logic with AwaitChild, but now that I think of it, it is just some badly written manual inlining of the semantics of Await::ONCE.
  • It evaluates into the return value of the generator, which is exactly the value that gets passed into the second parameter (callable $onComplete) of Await::g2c.

So here comes to your question: What does return yield $generator; mean? It just waits for the generator to complete, evaluate to its return value, then return its return value.

from await-generator.

How does the holder keep track of when either of those will be executed later? At this point yield Await::ONCE is executed, and the Generator is waiting for a response from the holder. It just hook the callable (that were passed in the first yield) with the send() and then let it do it's job?
This is the more complicated part of the internals. There are two cases:

• If the callback is called before ONCE is yielded, we store the result in the Await object and return it right away when ONCE is yielded
• If ONCE is yielded before the callback is called, the callback will check that the generator is already paused, so we can resume it immediately by calling send().

from await-generator.

I would like to understand the concept, but apart from the fact that the code looks surely more linear, I still couldn't transfer some minutes of reading Generator documentations to what I have to write. I couldn't find anything related to the yield in place of a callable, as a parameter. What does that supposed to mean if the meaning of yield is to (roughly) "pause the script"? And how does the return yield Await::something work? A Generator can return!? I also couldn't find something related to return yield.

I would like to understand more about the concept, I think the code style suits my need well. But aside from applying roughly what was on the README to use the libasyncql, I don't know how to understand anything else. Maybe I miss something... But for my lack of knowledge, please lay out things a little bit easier to cross-reference with the Generator PHP documentation.

from await-generator.

That answered my curiousity well! I'm just shy of kinda understanding the latter matters about AwaitChild. So take example the code snippet you provided (Please pardon some technical misunderstanding of mine):

function asyncSelect(string $query, array $args) : Generator {
  $this->connector->executeSelect($query, $args, yield, yield Await::REJECT);
  return yield Await::ONCE;
}

The Await::g2c takes care of holding this Generator. It run the executeSelect(), then the return yield Await::ONCE is run, which yield Await::ONCE is run first, then the holder just patiently waits for either the yield or the yield Await::REJECT to be executed. If it was yield, then it gives something (In this case, the $rows of the query?) to the holder, a result is received on the holder's side, then the holder send it back to the return yield Await::ONCE to be kindof return $rows, then this line would make sense:

$rows = yield $this->asyncSelect("query1", []);

My question is, if executeSelect() is run with the yield as an argument, something is sent back for the Generator, particularly executeSelect(), to continue executing, right? And if that then both of the arguments were passed 2 callables. How does the holder keep track of when either of those will be executed later? At this point yield Await::ONCE is executed, and the Generator is waiting for a response from the holder. It just hook the callable (that were passed in the first yield) with the send() and then let it do it's job? Everything would just freeze right there until the callable is called, is that right?

The holder understands the empty yield, and take that as a "promise" that promises to produce a result to send to the Await::ONCE? That means I can use multiple empty yield and that would means if I want to receive all of the results, I use Await::ALL, right? Now if that's right, then the results must be tracked, and all of them must be completed for the Await::ALL to be resolved. How does such tracking mechanism work?

from await-generator.

I forgot to mention, empty yield is an alias for yield Await::ONCE.

from await-generator.

Also, although yield Await::ONCE is the main focus for beginners to await-generator, it should only be used much less often. Await-generator is more like a framework than a library. Await-generator is only useful when all async logic make use of it.

However note that each generator function call allocates variables on the heap. Do not use heavily in hot paths. Await-generator assumes use cases to be useful only when async is heavily involved, and the performance tradeoff is only negligible when some scheduling has to be involved with the callbacks used anyway.

from await-generator.

https://sof3.github.io/await-generator/master

from await-generator.