tweag / distributed-closure Goto Github PK
View Code? Open in Web Editor NEWSerializable closures for distributed programming.
License: BSD 3-Clause "New" or "Revised" License
Serializable closures for distributed programming.
License: BSD 3-Clause "New" or "Revised" License
StaticApplicative
typeclass is currently defined as:
class StaticApply f => StaticApplicative f where
staticPure :: Typeable a => a -> f a
My suggestion is to change the signature to
class StaticApply f => StaticApplicative f where
staticPure :: Typeable a => Closure a -> f a
My reasoning is that, the biggest point of Static*
typeclasses are to represent composition of the values that can be distributed over the network. However, since that a
can not be serialised, it constrains the use of StaticApplicative
typeclass to only work on local values.
I can understand that probably this class is built for a specific purpose and that constraint wasn't necessary in that case; however I'd argue that having Closure
there is less restrictive than the current state, since one can always unclosure
it in the instance if a concrete a
is required.
My objective is to implement StaticApplicative
for the type defined here. dConstAggr
should be the staticPure
in case of my suggestion.
> /tmp/stackage-build8$ stack unpack distributed-closure-0.3.0.0
Unpacked distributed-closure-0.3.0.0 to /tmp/stackage-build8/distributed-closure-0.3.0.0/
> /tmp/stackage-build8/distributed-closure-0.3.0.0$ runghc -clear-package-db -global-package-db -package-db=/home/stackage/work/builds/nightly/pkgdb Setup configure --package-db=clear --package-db=global --package-db=/home/stackage/work/builds/nightly/pkgdb --libdir=/home/stackage/work/builds/nightly/lib --bindir=/home/stackage/work/builds/nightly/bin --datadir=/home/stackage/work/builds/nightly/share --libexecdir=/home/stackage/work/builds/nightly/libexec --sysconfdir=/home/stackage/work/builds/nightly/etc --docdir=/home/stackage/work/builds/nightly/doc/distributed-closure-0.3.0.0 --htmldir=/home/stackage/work/builds/nightly/doc/distributed-closure-0.3.0.0 --haddockdir=/home/stackage/work/builds/nightly/doc/distributed-closure-0.3.0.0 --flags=
Configuring distributed-closure-0.3.0.0...
> /tmp/stackage-build8/distributed-closure-0.3.0.0$ runghc -clear-package-db -global-package-db -package-db=/home/stackage/work/builds/nightly/pkgdb Setup build
Building distributed-closure-0.3.0.0...
Preprocessing library distributed-closure-0.3.0.0...
[1 of 3] Compiling Control.Distributed.Closure.Internal ( src/Control/Distributed/Closure/Internal.hs, dist/build/Control/Distributed/Closure/Internal.o )
src/Control/Distributed/Closure/Internal.hs:32:1: warning: [-Wunused-imports]
The import of ‘GHC.Fingerprint’ is redundant
except perhaps to import instances from ‘GHC.Fingerprint’
To import instances alone, use: import GHC.Fingerprint()
src/Control/Distributed/Closure/Internal.hs:61:1: warning: [-Wredundant-constraints]
• Redundant constraint: Typeable a
• In the type signature for:
fromDynClosure :: Typeable a => DynClosure -> Closure a
src/Control/Distributed/Closure/Internal.hs:134:1: warning: [-Wredundant-constraints]
• Redundant constraint: Typeable a
• In the type signature for:
cap :: Typeable a => Closure (a -> b) -> Closure a -> Closure b
[2 of 3] Compiling Control.Distributed.Closure ( src/Control/Distributed/Closure.hs, dist/build/Control/Distributed/Closure.o )
src/Control/Distributed/Closure.hs:48:1: error:
• Potential superclass cycle for ‘Static’
one of whose superclass constraints is headed by a type variable:
‘c’
Use UndecidableSuperClasses to accept this
• In the class declaration for ‘Static’
I am implementing distributed reference counting. These references require a non-trivial calculation to when serialized and de-serialized – a serialized reference still counts as reference, and thus the reference count must be increased and decreased respectively. Copying a serialized reference must also increase the reference count.
I wonder: How does this work with distributed-closure
? It seems that Encoded
values are copied...
There is a Static
instance for 2-tuples:
instance (Static c1, Static c2, Typeable c1, Typeable c2, (c1, c2)) => Static (c1, c2) where
closureDict = static pairDict `cap` closureDict `cap` closureDict
Please add respective instances for longer tuples as well.
The following program
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StaticPointers #-}
{-# LANGUAGE UndecidableInstances #-}
import Data.Constraint
import Data.Typeable
import Control.Distributed.Closure
import GHC.StaticPtr
data T a = T
deriving Show
type family F a where
F Int = Int
F a = a
class C a where
m :: Maybe a
instance C Int where m = Just 0
instance C (F a) => C (T a) where
m = Just T
instance (Typeable a, Static (C (F a))) => Static (C (T a)) where
closureDict = closure (static g) `cap` closureDict
-- closureDict = closure (static g :: StaticPtr (Dict (C (F a)) -> Dict (C (T a)))) `cap` closureDict
g :: Dict (C (F a)) -> Dict (C (T a))
g Dict = Dict
main :: IO ()
main = print (m :: Maybe (T Int))
fails with
tf.hs:30:17: error:
• Could not deduce (Typeable c0) arising from a use of ‘cap’
from the context: (Typeable a1, Static (C (F a1)))
bound by the instance declaration at tf.hs:29:10-59
The type variable ‘c0’ is ambiguous
• In the expression: closure (static g) `cap` closureDict
In an equation for ‘closureDict’:
closureDict = closure (static g) `cap` closureDict
In the instance declaration for ‘Static (C (T a))’
tf.hs:30:33: error:
• Couldn't match type ‘a0’ with ‘a’
Expected type: Dict c0 -> Dict (C (T a))
Actual type: Dict (C (F a0)) -> Dict (C (T a0))
• In the body of a static form: g
In the first argument of ‘closure’, namely ‘(static g)’
In the first argument of ‘cap’, namely ‘closure (static g)’
• Relevant bindings include
closureDict :: Closure (Dict (C (T a))) (bound at tf.hs:30:3)
tf.hs:30:42: error:
• Could not deduce (Static c0) arising from a use of ‘closureDict’
from the context: (Typeable a1, Static (C (F a1)))
bound by the instance declaration at tf.hs:29:10-59
The type variable ‘c0’ is ambiguous
These potential instance exist:
instance (Typeable a, Static (C (F a))) => Static (C (T a))
-- Defined at tf.hs:29:10
• In the second argument of ‘cap’, namely ‘closureDict’
In the expression: closure (static g) `cap` closureDict
In an equation for ‘closureDict’:
closureDict = closure (static g) `cap` closureDict
However it succeeds if the type signature of static g
is added as in the commented line. Unfortunately withStatic
produces the version with no type signature.
Both variants look correct to me. I've managed to reduce the example to this:
{-# LANGUAGE ConstraintKinds #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StaticPointers #-}
{-# LANGUAGE UndecidableInstances #-}
{-# LANGUAGE UndecidableSuperClasses #-}
import Data.Constraint
import Data.Typeable
import GHC.StaticPtr
class c => Static c where
closureDict :: Closure (Dict c)
data Closure a where
CStaticPtr :: StaticPtr a -> Closure a
CAp :: Closure (a -> b) -> Closure a -> Closure b
data T a = T
deriving Show
type family F a where
F Int = Int
F a = a
class C a where
m :: Maybe a
instance C Int where m = Just 0
instance C (F a) => C (T a) where
m = Just T
instance (Typeable a, Static (C (F a))) => Static (C (T a)) where
closureDict = CStaticPtr (static g) `CAp` closureDict
g :: Dict (C (F a)) -> Dict (C (T a))
g Dict = Dict
main :: IO ()
main = print (m :: Maybe (T Int))
If StaticPtr
is removed from the CStaticPtr
constructor and static g
is replaced with g
, then the program builds fine.
Could you kindly add a few examples of sending a closure or a plain function over the network and a process listening to that, accepting the function and then applying that function to some data? It would be really beneficial to have a few examples to show the entire flow of
Node 1 Node 2
---------- -----------
function --------------------------->
send accept the function and
apply it to a dataset
If I understand correctly, in a Haskell program, different modules can use static
on their own top-level variables to get StaticPtr a
where a
is the type of the top-level variable. Then a StaticKey
can be obtained from the StaticPtr
. Later, with a StaticKey
, a lookup can be performed using unsafeLookupStaticPtr
and then deRefStaticPtr
to get back a value of type a
.
When I write a program using static pointers, e.g. to implement something like RPC, I would typically be getting the StaticKey
s from some external source, e.g. disk or network, and then trying to get a value using unsafeLookupStaticPtr
. The value I get from unsafeLookupStaticPtr
could be unsafe for 2 reasons: 1. The type assumed is wrong. 2. The type assumed is correct but the value is some function from some module (e.g. from some hackage package) that I didn't even know 'exposed' a StaticPtr
in the static table. If I inadvertently run that function, bad, unexpected, things may happen. I think the key problem is that any module can make its own static pointers and GHC does not mandate that only the main program is allowed to do so.
Some other libraries (e.g. Control.Distributed.Static
) avoid the the second problem by requiring the explicit use of a table during the 'lookup' (RemoteTable
). It seems that distributed-closure is using the Haskell static table directly. Is it susceptible to the second type of safety problem? If so, what can I do to restrict the functions that I can get from unclosure
?
Please make a new Hackage release. There has been quite a bit of changes since the last release.
I find that I need this in my code when I have closures that create closures:
instance Static c => Static (Static c) where
closureDict = closureDict
Instances declared with StandaloneDeriving
are not recognized by withStatic
, and they don't get their static version defined. The following program does not compile (missing instance for (Static (Show D))
).
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StaticPointers #-}
{-# LANGUAGE StandaloneDeriving #-}
{-# Language TemplateHaskell #-}
import Control.Distributed.Closure
import Control.Distributed.Closure.TH
import Data.Typeable
data D = D
deriving (Typeable)
withStatic [d|
deriving instance Show D
|]
showC :: forall a. (Typeable a, Static (Show a)) => Closure a -> String
showC a = unclosure $ static (\Dict -> show)
`cap` (closureDict :: Closure (Dict (Show a)))
`cap` a
main :: IO ()
main = do
putStrLn $ showC $ static D
Neither does withStatic
work with derived instances
withStatic [d|
data D = D
deriving (Typeable, Show)
|]
-- Static (Show D) doesn't get defined
As a result, there is no way to auto-generate static versions of instances with default implementations.
The build fails with the following error:
examples/ClientServer.hs:83:1: error:
• Illegal instance for a type synonym
A class instance must be for a class
• In the instance declaration for ‘Serializable SerializableInt’
|
83 | withStatic [d|
| ^^^^^^^^^^^^^^...
In order to fix it I have to define some instances manually which seemed to work.
When compiling tests on GHC 8.6, we get below error:
tests/test.hs:31:1: error:
• Illegal nested constraint ‘Static (Show (F a))’
(Use UndecidableInstances to permit this)
• In the instance declaration for
‘Static (Show (T1 a_a8ni b_a8nj))’
|
31 | withStatic [d|
| ^^^^^^^^^^^^^^...
tests/test.hs:31:1: error:
• Illegal nested constraint ‘Show (F a)’
(Use UndecidableInstances to permit this)
• In the instance declaration for ‘Show (T1 a_a8ni b_a8nj)’
|
31 | withStatic [d|
| ^^^^^^^^^^^^^^...
The offending line is this one:
withStatic [d|
...
instance Show (F a) => Show (T1 a b) where show = undefined
...
|]
where F
is a type family. The change mentioned here is probably the cause: https://ghc.haskell.org/trac/ghc/wiki/Migration/8.6#UndecidableInstancesispickier
Removing the offending instance or setting -XUndecidableInstances
solves the issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.