asynkron / protoactor-go Goto Github PK
View Code? Open in Web Editor NEWProto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
Home Page: http://proto.actor
License: Apache License 2.0
Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
Home Page: http://proto.actor
License: Apache License 2.0
Go has supervision, but we need to set the semantics around this.
Currently Tell
does not pass any Sender to the target actor.
Ask
on the other hand does, so that the asked actor can reply.
There are cases where one want to do an async Ask, that is, send a message with a sender, so the target actor can reply, but without blocking/waiting in the caller actor, where the response is just sent to the asking actor as a message.
Ask
is a heavy operation, it will register a temp actorref in the process registry so that is not optimal for this kind of async operation.
How should we make it possible to Ask without a result, or Tell with a sender?
AskAsync(msg, sender)
TellWithSender(msg, sender)
Other suggestions?
cc @dtravin
ReceiveTimeout is implemented in Go, it needs to be ported over to C# too
The build scripts are currently windows only and most likely, coupled to my personal machine.
This needs to be cleaned up
We don't do any validation at all of the name passed to Spawn.
We should probably prevent the use of things like /
as that is used when creating child names.
hi,
I'm a newbie in Golang, trying to build a chat service with gam,
just wonder if is there something similar to Distributed Publish Subscribe in Akka
Now that VS 2017 RC is out, it should be fairly easy to port the C# implementation to .NET core
Similar to RoundRobin routers, but with a random selector instead of sequential.
I would like to see that we come up with some good defaults that we all can agree on being the right choice for various parts of actors.
What do we want, Kill after x restarts or exponential back-off ?
Killing makes sense in the way that having broken actors floating around is pointless.
Exponential back-off makes sense in the way that many errors are transient and just waiting might solve them.
We currently use the unbounded mailbox, unbounded queues are notoriously bad for systems at scale.
IMO, we should introduce a new queue, built on the same backing structure as the unbounded, but drops messages to deadletter on overflow.
So if the bounds of this queue is 1000, then when you try to post a message to it when length is > 1000, drop the message.
(coming up with a good default for that queue size might be hard though)
One could think that blocking would be a better choice, which it might be inprocess. but if the Remote endpoint reader tries to post to a full queue and blocks, you are now blocking the endpoint reader.
Having a bounded mailbox as the default is a no-go as that is blocking on overflow, and consumes a lot of memory all the time.
Other areas to consider on this topic?
Suggestion
routing.Spawn(host, name, kind)
Becomes:
routing.Spawn(host, kind)
routing.SpawnNamed(host, name, kind)
Which would be more consistent with actor.Spawn
The C# implementation lacks the supervision support. this needs to be copied from Go
I am using your package to write a little chat program similar to your example.
Now, when one of the clients disconnects from the server (simple exit or crash) and the server tries to send something to the disconnected client the endpointWriter
throws a fatal error which cannot be catched.
Did I miss something in remoting
to manage errors or checking if an remote actor got disconnected?
GAM Cluster needs two ports per node, one for gRPC, and one for the Hashicorp Memberlist.
Currently, this assignment is a hack, the gRPC port is simply assigned from memberlistPort+1.
Which obviously will not be a stable strategy for production usage.
We need a better way of dealing with this.
Hi, i have one question about routing group. I was studying how the routers are written since I wanted to help with consistent hashing issue.
Just want to ask what is the intention about group routers. Are they static by any purpose? You have mentioned consul.io in readme. That is exactly what i want to use for clustering. But how to incorporate it to gam ?
Only way currently is to create front facing structure that will implement Tell(message interface{}) and forward messages to some group router. If anything changes in consul network this struct will re-create group router and stop old remote pids. Is that something that you are proposing? Or i have missed something.
It would be awesome if group routers would have ability to change routees in runtime.
Trying to call pid.Stop() fails on local actors with an "Unknown host" exception.
I managed to "fix" it by modifying TryAdd in ProcessRegistry.cs:
public ValueTuple<PID, bool> TryAdd(string id, ActorRef aref)
{
var pid = new PID()
{
Id = id,
Ref = aref, //cache aref lookup
Host = "nonhost" // local <---- This fixes the error
};
var ok = _localActorRefs.TryAdd(pid.Id, aref);
return ValueTuple.Create(pid, ok);
}
Maybe I am missing something here?
Sending a single message to multiple remote PIDs can consume potentially large amounts of memory and CPU resources. We propose to add support for batching these messages so they can be serialized and deserialized once per node.
Group routers allow the user to add arbitrary PIDs to receive messages based on the routing strategy of the router. These PIDs could reside on many remote nodes. Certain scenarios allow a single message to be forwarded to all of the PIDs, which can be very inefficient if many of them are remote. In the case of a broadcast router, messages are always forwarded to all the routees. In addition, a router that receives a router.Broadcast
message will forward the inner message to all its routees.
As an example, a group broadcast router with 1,000 PIDs residing across 10 remote nodes would result in a single message being serialized and deserialized 100 times on each node. In addition, the source node will have to serialize the message 1,000 times. This equates to a potentially large increase in memory and CPU resource usage on all involved nodes.
We propose creating a new message which represents a broadcast of an inner message to a set of PIDs on a remote node:
message BroadcastEnvelope {
string typeName = 1;
bytes messageData = 2;
repeated string IDs = 3;
}
The typeName
and messageData
represent the serialized protobuf message that will be reconstituted at the remote and delivered to the local PIDs. The IDs
array is the Id
component of the PID
, given the Address
is redundant.
An actor on the remote node will deliver the inner message to all PIDs listed in the IDs
array.
Add Tell
method to PIDSet
, allowing efficient broadcast of messages to a set of PIDs
Is there a Pipe method so that an actor can send messages for itself.
If I send a message to the actor from a goroutine or from another actor it works, but if I want to send a message from itself the message does not arrive.
Thanks
hi bro. i'm thinking about using gam in my project. and i need the sender field of deadletters object.
do you have and plan to add this field? or i can make a pull request myself if you are busy.
HI,
I have a very simple test program:
package main
import (
"github.com/rogeralsing/gam/actor"
"fmt"
)
type PublishActor struct {
AppPath string
RabbitPid string
}
func (state *PublishActor) Receive(context actor.Context) {
switch msg := context.Message().(type) {
case actor.Started:
println("--> Publish Actor Started")
case QueueParams:
state.AppPath = msg.AppPath
state.RabbitPid = msg.RabbitPid
println(fmt.Sprintf("Publish Actor Params = %s\n", state.AppPath))
default:
println(fmt.Sprintf("msg = %v\n", msg))
}
}
type QueueParams struct {
AppPath string
RabbitPid string
}
func main() {
props := actor.FromInstance(&PublishActor{})
publishPid := actor.Spawn(props)
qp := QueueParams{"Path", "RabbitPid"}
publishPid.Tell(qp)
}
The problem is that the QueueParams message does not received. Only the Started Message is received by the PublishActor,
The console output is this:
--> Publish Actor Started
What Am I doing wrong ?
Thanks
We already have fairly good support to monitor stuff with the Middlewares and MailboxStatistics.
It would be useful to be able to trigger alarms when mailboxes grow out of control.
That could be implemented as a MailboxStatistics.
Thoughts?
The description of dependencies in readme and updateproto.bat is outdated,and building project is hard for newbie。
So, recommended use govendor to manage dependency.
We should promote good patterns as far as possible, and provide easy to use defaults.
One such thing might be some form of rate-limiter / throttling middle-ware.
The most naive form could easily be implemented as a middleware that simply sleeps after each message received.
This would be easy do to in .NET too as we are 100% async wait there, so we could use await Task.Delay w/o blocking any threads.
Throttling can ofc be implemented in infinitely complex ways with x messages during a timewindow etc.
But some sensible patterns would be nice
In order to support TLS, we need to expose some of the gRPC internals to the user.
This is largely already done when starting the remote infrastructure.
But there might be a need to expose gRPC channel factories so the user can customize the entire process.
See http://www.grpc.io/docs/guides/auth.html for more info on gRPC auth.
Thoughts?
Hi everyone,
My name is Nicolas, I'm a french .NET/python programmer. And for about 3 weeks a little Go programmer.
I would like to know if it's possible to contribute to the project ?
Thank you.
I would like to use this thread to set the design goals for the project.
Here are the ones that was set from day 1, slightly re-phrased
Minimalistic API - The API should be small and easy to use. Avoid enterprisey JVM like containers and configurations.
Build on standards - Do not try to reinvent solved problems, build on proven tech and standards.
Protobuf for serialization, gRPC Streams for network transport, Consul for cluster membership
Message passing, not object passing - Serialization is an explicit concern, don't try to hide it.
Be fast - Do not trade performance for magic API trickery.
Later additions:
Cross platform - do not rely on platform specific functionality, this is enabled by the "build on standards"
Offload secondary concerns - offload features like logging, configuration, scheduling, dependency injection to 3rd party libraries. they are not part of the core problems solved by the framework.
Add .NET build pipeline in Travis. possibly using Cake build
Hi,
I am interested in using proto-actor in my "system", as Akka cannot satisfy my needs of handling cross-platform actors. My system would be developed as a plug-in model allowing to create plugins as actors that could be written in different languages. The core would be in C#, and GO i have out of the box. What about Java,Python,C++ ? Is there a support plan for it?
Implement the same functionality found in Akka ConsistentHash routers.
Messages should either be hashed and sent to a routee for that hash.
I've created some initial work an an exponential backoff supervisor.
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/exponential_backoff_strategy.go
This needs more work, config options and tests.
Routers are currently missing from the C# implementation
currently, there is no way to hook into the gRPC connection or options when starting GAM remoting.
There should be some way to start and handle request on existing gRPC connections.
This should open up for authentication and secure connections
IMO, we should rething the naming on the new mailbox package.
props.WithMailbox(mailbox.NewUnboundedProducer())
This reads a bit strange to me, not sure what would be better though.
props.WithMailbox(mailboxUnboundedProducer())
props.WithMailbox(mailbox.NewUnbounded())
props.WithMailbox(mailbox.Unbounded())
Or just stick with what we have?
In tests there is need to wait until actor is fully stopped.
Hi,
I am starting with gam. I have a good experience in Akka (Scala) and Akka.net.
I have an actor that can recieve more than one type of message, like this:
func (state *PublishTransactionsActor) Receive(context actor.Context) {
switch msg := context.Message().(type) {
case actor.Started:
fmt.Println("Start Publish Actor")
state.Channel = nil
state.Queue = nil
case messages.QueueParams:
fmt.Println("Setup Publish Actor Params")
state.Channel = msg.Channel
state.Queue = msg.Queue
case messages.TransactionsAvaiable:
fmt.Printf("Transactions = %d\n", msg.NumOfTransactions)
}
}
I am sending the message like this:
qp := messages.QueueParams{rabbitChannel, q, ack, nack}
publishPid.Tell(qp)
When I put a break in the switch line I can see that message QueueParams is arriving but it does not enter in the case.
What am I doing wrong ?
Thanks
Same as One for One strategy but apply the directive to all children.
At the moment, the dispatcher functionality is not independent of the mailbox. Mailbox is responsible for checking the throughput and yielding, which limits the variety of dispatcher that can be used. Also, some dispatchers might require further access to the processing message loop. e.g. a dispatcher might map an actor to a single goroutine without letting it go, or need to process stuff before the goroutine terminates.
My suggestion is to separate the dispatcher functionality from the mailbox, by introducing new functions under the Dispatcher interface and sprinkle those functions in related places in the mailbox code. You can check my suggested approach here: Huawei-CPLLab@7740008
Also, as @rogeralsing suggested it might be better to put back the index
variable in the mailbox and pass it to BeforeProcessingMessage
function, to make the dispatcher stateless, and thus avoid creating a dispatcher per actor.
On the same note, it might be even better if we can reuse the interception points in MailboxStatistics to achieve this.
The supervisor strategies should terminate the failing actor after enough failures have occurred in the configured timespan.
We need to track PID failures in a map inside the actor cell.
Currently the EndpointReader will just issue a pid.Request(..)
in the EndpointReader loop.
This means that even if you send a system message such as Watch
, the message would be delivered through the user message pipeline. which is incorrect.
In order to fully support passing contextual information the current PID.Tell
model will not work.
One suggestion is to move the Tell
operation over to the actor Context.
e.g.
//instead of
pid.Tell(msg)
//we would instead have
ctx.Tell(pid, msg)
This would allow us to build things like tracing using e.g. Zipkin.
It would also greatly simplyfy having multiple actor systems running at the same time. e.g. for parallel testing.
ctx would then be like a phone and the PID would just be the phone number.
Currently PID is both.
This would be a major breaking change, so I'm not completely sold on this idea myself, but I do see the added value it brings
Hi,
I am planning to migrate a Scala Realtime Transaction Tracking to Go using Gam. I need a faster and more concise application.
I am using Gam and it is working perfectly, Is Gam ready for production ?
Thanks
Currently if you spin up a grain, and the grain is idle for a long time, nothing happens.
It would be nice if we could passivate the grain so that it unloads from memory.
There are a few problems that needs to be solved for this:
Languages and frameworks like Erlang/Elixir, Akka, Akka.NET, ServiceFabric and Microsoft Orleans, are all incompatible. there is no actor native way of communicating between the different stacks.
(You can ofcourse integrate using HTTP or via message queues, but this is not the question asked here)
GAM is built from day one to support cross platform actors.
We use gRPC Streams for remote communication, Protobuf for serialization.
There is no fancy API tricks or strange local thread state required.
It is all standard technologies composed together.
The TLDR; description is that it is gRPC services with actor infrastucture behind the endpoints.
Is there any serious interest in having fully cross platform actors, both Akka and MS Orleans style virtual actors for other platforms?
If so, what platforms should we focus on?
e.g. Is there an interest of having distributed and virtual actors for Rust?
Is there an interest to be able to integrate clustered .NET software with Go?
For those new to GAM, here is a quick recap of features offered.
See: https://github.com/AsynkronIT/gam#go-actor-model
Become
semanticsWhat I am trying to get at here is that:
Sorry,
But I could not find how to get the current actor pid. I have tried with actor.Context.Self():
pid := actor.Context.Self()
but I got the error:
not enough arguments in call to method expression actor.Context.Self
But the definition of actor.Context.Self defines that it wll return *PID.
Can you help me
make
go build ./...
package ./mailbox
imports github.com/AsynkronIT/protoactor-go/internal/queue/goring: use of internal package not allowed
package ./mailbox
imports github.com/AsynkronIT/protoactor-go/internal/queue/mpsc: use of internal package not allowed
package ./remote
imports github.com/AsynkronIT/protoactor-go/internal/queue/goring: use of internal package not allowed
package ./remote
imports github.com/AsynkronIT/protoactor-go/internal/queue/lfqueue: use of internal package not allowed
make: *** [build] Error 1
Install process is entirely broken.
go version
go version go1.6.3 linux/amd64
protoc version
libprotoc 3.0.0
Your library cannot be installed with go get
and trying to build it by hand yields the following error
cd actor/; protoc --gofast_out=plugins=grpc:. --proto_path=. --proto_path=/home/jakob/projects/other/lib/src ./*.proto
go build ./...
# github.com/rogeralsing/gam/remoting/messages
remoting/messages/protos.pb.go:93: undefined: grpc.SupportPackageIsVersion2
make: *** [Makefile:7: build] Error 2
I'm currently working on MS Orleans style virtual actors.
This is implemented the same way as they do in Orleans.
This allows you to do massively distributed systems without having to care where and how actors are created.
func main() {
cluster.Start("127.0.0.1:0", "127.0.0.1:7711")
//get a PID to your virtual actor, the cluster module will find or register it's existence for you
//on one of the nodes in the cluster.
pid := cluster.Get("myfirst", shared.Type1)
pid.Tell(&shared.HelloMessage{})
console.ReadLine()
}
In Akka, there is the ActorSystem.Scheduler
In Proto.Actor, I would rather like to see some Scheduler interface, where we can provide different kinds of implementations.
e.g. in mem, cron job based etc.
Users can then either inject the scheduler they want into their actors, or, just have a package wide instance of whatever scheduler they want to use.
If we do this as part of the Proto.Actor library, I would like to see this as a separate package to keep the API surface as minimal as possible in the core actor package.
Hi,
Would you please give details about protoactor cluster.
Thanks
Provide a new API for the cluster package to send an asynchronous message to a set of named, homogeneous actors.
cluster.TellMany(names []string, kind string, message interface{})
Two possible approaches to the implementation, with different pros / cons.
Pros
Cons
Pros
Cons
Option 3 is a variation of Option 1, providing benefits of both Option 1 and 2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.