virtuslab / avocado Goto Github PK
View Code? Open in Web Editor NEWSafe compile-time parallelization of for-comprehensions for Scala 3
Home Page: https://virtuslab.github.io/avocADO/docs/index.html
License: Apache License 2.0
Safe compile-time parallelization of for-comprehensions for Scala 3
Home Page: https://virtuslab.github.io/avocADO/docs/index.html
License: Apache License 2.0
Write tests for:
The following example should work correctly:
val run: IO[Int] =
ado {
for {
a <- wait.map(_ => 1)
b <- wait.map(_ => 2)
c <- wait.map(_ => a + 2)
d <- wait.map(_ => 4)
(e, f) = (1 -> 2)
(g: Int) = 3
2 = 2
cc@C(i) = C(5)
obj.AClass[Int](t) = obj.AClass(1)
obj.AClass[Int](1) = obj.AClass(1)
} yield c + b
}
Also, look at this for more possible bindings.
Libraries to consider:
avocADO
should be able to correctly handle if
guards in for
comprehensions.
e.g.
for {
a <- Some(1)
if a < 2
b <- Some(2)
} yield b
It's possible that the parallelize
macro might also benefit from having a version (or flag) that removes unneeded trailing map
calls, similarily to normal fors as described in #115.
As mentioned here: https://contributors.scala-lang.org/t/for-comprehension-requires-withfilter-to-destructure-tuples/5953/3
-source:future
makes some changes to desugaring of for
comprehensions. Specifically e.g. desugaring of classes/tuples with binds do not use withFilter
now.
avocADO
should be able to correctly handle given
bindings in for
comprehensions, both as pure statements and as bindings.
e.g.
def getImplicit(using i: Int): Option[Int] = Some(i)
for {
given Int <- Some(2)
b <- getImplicit
} yield b
def getImplicit(using i: Int): Option[Int] = Some(i)
for {
_ <- Some(2)
given Int = 1
b <- getImplicit
} yield b
You can unify both cats-effect-2
and cats-effect-3
modules into a single "cats" module (doesn't actually need cats-effect, and opens the usage to a lot more datatypes than just effect types). Something like :
package avocado.instances
import avocado.AvocADO
import cats.Monad
import cats.Parallel
object cats {
given [F[_]: Monad Parallel] : AvocADO[F] = new AvocADO[F] {
def pure[A](a: A): F[A] = Monad[F].pure(a)
def map[A, B](fa: F[A], f: A => B): F[B] = Monad[F].map(fa)(f)
def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] = Parallel.parTuple2(fa, fb)
def flatMap[A, B](fa: F[A], f: A => F[B]): F[B] = Monad[F].flatMap(fa)(f)
}
}
This will work for both cats-effect-2 and cats-effect3, and only needs the dependency to cats-core. Moreover, it'll give users access to the ado
block in contexts such as Either/Validated, where errors would automatically be aggregated in a data structure.
We might want to provide a macro that removes the trailing map if it is unneeded. The use for it would be to make a for
-comprehension style expression stack-safe, since flatMap
can be (and often is) implemented in a stack-safe manner, but a map
isn't. One such example is:
def loop: IO[Unit] =
for
_ <- putStrLn("fix this pls")
_ <- loop
yield ()
We might want to add a macro elimMap
that would be used like so:
def loop: IO[Unit] =
elimMap {
for
_ <- putStrLn("fixed")
_ <- loop
yield ()
}
That will remove the trailing map call if it is unnecessary and give an error (or warning) if the map
changes the result.
For trivial types AvocADO
should be derivable.
Types that should be derivable:
Hello Dear Maintainers, thank you for your work on this incredibly useful library!
I would like to propose an improvement in order to use a more intuitive name that directly relates to the functionality that the library provides.
I understand the origin of the name and its references to Haskell. However, from the perspective of Scala, not everyone has such roots, and without additional explanations, the name ado
doesn't say much. A new person in the project, seeing something like this in the code, may feel lost. Seeing the word parallel
or parallelize
, they will easily understand what a given code block does.
If this is a good idea and is well received, creating an alias would likely be a solution, providing backward compatibility. Taking it a step further, standardising and adding the @deprecated
annotation to ado
would be an additional option to consider.
I would also refer to @szymon-rd's presentation from Scalar 2023, which teaches us that we may want our code to be easier to read and libraries designed to be intuitive:
https://www.youtube.com/watch?v=nA9gRGpOOJc
Finally, an example for applicative-for would look like that (using parallel
, parallelize
, or similar).
val run: IO[Int] =
parallel {
for {
a <- doStuff1
b <- doStuff2(a)
c <- doStuff3
} yield combine(a, b, c)
}
What do you think?
Currently map
s and ``flatMaps used in the resulting transformation are called as methods on the composite expressions. This makes calling them a potentially error-prone process. If we choose to extend that
avocado.Applicative` typeclass to also include those functions it would make it easier to generate the correct code. (This would also require a name change for the typeclass though, since it will no longer be "just" applicative)
Cats applications can often use the following pattern:
def doStuff[A, F[_]: Monad](fa: F[A]): F[Int] =
for {
a <- fa
b <- fa
c <- fa
} yield 1
Which works because of implicit syntax classes. i.e.
import cats.syntax.all.*
Those types of for
comprehension should be handled correctly.
Currently avocADO
parallelizes the following comprehension:
for {
a <- doStuff1
b <- doStuff2(a)
c <- doStuff3
} yield combine(a, b, c)
to sth conceptually like:
for {
a <- doStuff1
(b, c) <- doStuff2(a).zip(doStuff3)
} yield combine(a, b, c)
The reasoning behind it is that if one wants to parallelize comprehensions that have the effect monad abstracted away, the concrete monad at the end might use a sequential state. In that case, there can't be any parallelization done, because that would affect the end result of the program. In such cases, it should still be possible to use ado
, but just provide a different AvocADO
instance (with a sequential implementation of zip
).
Though for use cases that are more concrete, like programs based on god monads (ZIO etc.), a more aggressive parallelization should be available. One that can also reshuffle (change the order of) the expressions.
Concretely, it should be possible to convert our example (with a more aggressive strategy) to sth conceptually like:
for {
(a, c) <- doStuff1.zip(doStuff3)
b <- doStuff2(a)
} yield combine(a, b, c)
avocADO
should support SIP-62 changes (see implementation here: scala/scala3#20522)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.