Fiber / green threads / coroutines? about truffleruby HOT 7 CLOSED

@chrisseaton I don't believe coroutines (not continuations) "fundamentally don't match the execution model we are using at the moment". See my previous answer.
Using that patch would allow to have similar performance characteristics as MRI.
But yes, it's not practical to require a custom JVM build of course.
I agree it's better to be clear in compatibility: we don't support coroutines in the current distribution and use threads for Fibers.
But there is nothing fundamentally impossible here.

from truffleruby.

Thanks for your responses.

My use-case is related to celluloid and other programming libraries that hide asynchronous behavior behind a Fiber-based synchronous interface. At times there can be hundreds or thousands of fibers utilized by the libraries, so lightweight fibers are useful in that regard.

I would like to use TruffleRuby for a few proof-of-concept projects. I will likely need to implement some kind of fiber pooling mechanism to prevent run away thread allocation. If I discover anything interesting, I'll be sure to share it with you all.

from truffleruby.

Currently TruffleRuby uses Java threads for Fibers.

The standard JVM does not yet support native coroutines.
However, Lukas Stadler made a patch to add native coroutine support (http://ssw.jku.at/General/Staff/LS/coro/).

I am using that patch in my Oz language implementation.
The patched JVM is at https://github.com/eregon/jvmci and the corresponding API in https://github.com/eregon/truffle/tree/coro, which uses threads or coroutines depending on the JVM it runs on.

So we might use that to try what would be the benefits of native coroutines, but it is unpractical as it requires a custom JVM build.

Do you have any particular library or application in mind where it is important to have lightweight Fibers?
Thread switching is not so much slower, but they use more memory.

from truffleruby.

This is documented in our compatibility document. I've updated it a bit to say some more about fibers.

https://github.com/graalvm/truffleruby/blob/9d3a8f3e9f47915a0e4167e1002960a70b68f44b/doc/user/compatibility.md

Fibers, continuations and coroutines are all things that fundamentally don't match the execution model we are using at the moment and are unlikely to every be implemented with the same performance characteristics as MRI.

It's a shame, but it's one of a very few hard compatibility limitations we have.

from truffleruby.

To clarify, when I say 'fundamentally doesn't match our execution model' I mean running on an unmodified JVM. Benoit is talking about things which may be possible with modified JVMs or the SubstrateVM.

from truffleruby.

FWIW reusing FIber threads is something relatively easy, just not done currently in TruffleRuby (patch welcome!).
https://github.com/eregon/truffle/blob/247496f4984d9f08775f165cd7538a6d4c0dd90b/truffle/com.oracle.truffle.coro/src/com/oracle/truffle/coro/ThreadCoroutineSupport.java#L108

from truffleruby.

For information, Fibers now use a thread pool, so the cost to create many of them should be much lower.

from truffleruby.