Comments (6)
Hi Gareth,
I have an open issue for looking into Akka.Persistence in general with Phobos - it looks to us like we don't tie the Persist
before and after in together correctly. Do you have a timeframe you need this done by? We were going to work on that once we have our OTel tracing beta all the way out the door this month.
from phobos-issues.
Hi Aaron,
I wasn't sure if it was the same issue or a different one (albeit related), so I raised it separately. While we were waiting for an update, we put in a workaround for the other persistence issue which is to wrap the call to Persist and short circuit the trace manually as follows:
protected void PersistWithTracing<TEvent>(TEvent @event, Action<TEvent> handler)
{
if (@event is null)
throw new ArgumentNullException(nameof(@event));
var parentSpan = Context.GetInstrumentation().ActiveSpan;
Persist(@event, _ =>
{
if (handler is null)
return;
var phobosActorContext = Context.GetInstrumentation();
var spanBuilder = phobosActorContext.Tracer
.BuildSpan($"custom.akka.persistence.handler_{@event.GetType().Name}")
.AsChildOf(parentSpan);
// Get tags using reflection
var tags = GetTags(phobosActorContext.ActiveSpan);
foreach (var tag in tags)
spanBuilder.WithTag(tag.Key, tag.Value);
using (spanBuilder.StartActive())
handler(@event);
});
}
In some cases this is working, but in other it's not and the trace in the description is one example where it isn't. If this is all related though I'm happy for you to roll the two issues up and close one; however you best want to track it.
As to a timeframe, I'll discuss it with the team and get back to you as soon as I have some info on that.
Many thanks!
from phobos-issues.
Progress update on our end: we've added some prototype instrumentation to capture all actor startup / restart activity prior to processing messages. This is a necessary ingredient for being able to trace recoveries from the point of view of the PersistentActor
. The next stage is making sure that the persistence plugins themselves can properly correlate the activity on their end - this might require an update to Akka.Persistence itself to make the plugins more traceable, but we'll see.
from phobos-issues.
Thanks Aaron; appreciate the update!
from phobos-issues.
Whoops, not closed yet. Just merged in an Akka.NET PR that is needed to implement this in Phobos.
from phobos-issues.
Resolved via Phobos 1.4.0
from phobos-issues.
Related Issues (20)
- Lots of noise in logging when using cluster sharding HOT 6
- Span is created for akka.actor.ask even when create-trace-upon-receive is set to off HOT 5
- UsableContext value becomes NULL in the middle of processing chain HOT 2
- Stashing message may result in reset UsableContext HOT 16
- Create a gauge for all alive actors of each type in any given `ActorSystem` HOT 1
- How to filter `Ask<T>` out of traces? HOT 6
- Manually created trace spans don't seem to flow across the cluster HOT 14
- When spans are created manually messages from cluster singleton break trace transactions HOT 19
- Does sending a message to sender from Persist/PersistAsync break trace transaction? HOT 10
- Can auto-generated shard messages be omitted from a trace transaction with manual span management? HOT 18
- Need to make all Akka.Cluster.Sharding infrastructure transparent in traces HOT 12
- Missing `unit` for all metrics received from Phobos in Prometheus HOT 16
- `Tracer.CurrentSpan` and `PhobosActorContext.ActiveSpan` should be the same when actor is processing message, but currently are not
- Add support for tracking Akka.Cluster.Sharding `ShardRegion` metrics HOT 1
- Phobos `DeadLetter` logging HOT 1
- `Ask<T>` tracing does not properly follow `Tracer.ActiveSpan` HOT 1
- Bug: System.InvalidOperationException: ValueFactory attempted to access the Value property of this instance.
- Link to Release notes from Nuget package? HOT 3
- Error "Cannot find serializer with id [210] (manifest [WTRACE])" when remoting from an F# script HOT 5
- How to handle messages between actor system where one of them uses Phobos tracing? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from phobos-issues.