https://docs.rs/tracing/latest/tracing/

Discover any work required to ensure compatibility of the Observe SDK with the new breaking changes introduced by OpenTelemetry.

Will be blocked until #39

Goal is to add a set-metadata function on the tracer for Go and JS like we are doing in rust #38

Currently the observe api may be used in C++, but requires using raw pointers to pass strings. It would nice to be able to use std::string to pass strings instead.

Publish the observe #51 rust crate somewhere so people can do some manual instrumentation

Enable support for the Observe SDK to execute and profile components. Since wazero does not yet support components, this issue is scoped to (in priority order):

• Rust (involves upgrading to Wasmtime 13 - https://github.com/bytecodealliance/wasmtime/releases/tag/v13.0.0) Ticket: #142
• JavaScript via https://github.com/bytecodealliance/jco

To test, we'll have to compile a component that has been instrumented. We should be able to do this manually with the Rust Observe API

The Rust version has span_tags, but it looks like it was never added to the C api.

Similar to how we check for the min & maj version globals in the SDK for compatibility in Rust and Go (if instrumented automatically via the compiler) we should implement it in JS.

Move the planktonic repos to some subfolder in observe-sdk. It should get deployed when observe-sdk is deployed. It should have a readme and a little content about how it works. Maybe a short video.

As an Iota user, I would like to be able to dynamically swap the adapter used by Iota at request time, so that I have more flexibility when testing and demo'ing Observe Toolkit adapters

As this check would be advisory, it is not essential for #143 and thus #142 . Work is near done on the wasm-instr side, but split off here to avoid holding up component support.

• Add advisory check on version in wasm-instr producers section
  • Rust
  • Go
  • JS

On my machine many.rs never prints 250 times, usually it only print 200 times. However, it seems to work most of the time on CI.

test tests::integration_many ... FAILED

failures:

---- tests::integration_many stdout ----
thread 'tests::integration_many' panicked at 'assertion failed: `(left == right)`
  left: `200`,
 right: `250`', rust/tests/many_tests.rs:23:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:578:5
   1: core::panicking::panic_fmt
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/panicking.rs:67:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/panicking.rs:228:5
   4: many_tests::tests::integration_many
             at ./tests/many_tests.rs:23:9
   5: many_tests::tests::integration_many::{{closure}}
             at ./tests/many_tests.rs:12:30
   6: core::ops::function::FnOnce::call_once
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:250:5
   7: core::ops::function::FnOnce::call_once
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    tests::integration_many

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.65s

error: test failed, to rerun pass `--test many_tests`

The simplicity of the Observe API bindings makes it an ideal library candidate to implement as a single header library. A script can make it easy to still provide the traditional src file + header setup for users that prefer that setup.

• JS
• Go
• Rust

We should add errors on a span as a concept to the core Event data-structure. It should be mapped to sink equivalent concepts in the adapters.

In the updated adapter.start calls or wherever we're passing in options, we should take the value by reference vs. expecting to own it. particularly in Rust, currently each trace context must make its own new Options, or if cloneable, copy it from elsewhere.

e.g.

- let trace_ctx = adapter.start(_, _, options);
+ let trace_ctx = adapter.start(_, _, &options);

It could also be done in Go so the value isn't copied around, but that's more of a performance/resource optimization than a pesky DX as it is in Rust.

The testing situation is currently grim, mostly done by hand. A first start is to unit test the individual modules. We also need to simplistic and well understood modules that we can run and verify that the captured events match what we expected from the module.
This will involve building at least one of those modules, and making a list of others we'll need. Then well need a way to test that the modules are events are being generated correctly, however we should not let changes to the adapters break these tests.

See: https://github.com/dylibso/observe-sdk/blob/main/rust/src/adapter/otel_formatter/mod.rs#L33C1-L38

If it makes sense to go more generic than serde_json then let's do it, but for this adapter we at least need to support json, so the serde_json option is fine

The shutdown methods (e.g. https://github.com/dylibso/observe-sdk/blob/main/rust/src/adapter/otelstdout.rs#L97 ) should wait on the runtime instance to close it's side of the event channel. This does get tricky because we don't have a way to tell the runtime instance it needs to stop sending events. So the parts of this issue will be:

1. the Collector needs to signal the InstrumentationContext that we're shutting down even collection
2. A way for the InstrumentationContext to know that we are no longer collecting events, all future calls from the module should be noops (e.g. https://github.com/dylibso/observe-sdk/blob/main/rust/src/lib.rs#L135 )
3. The InstrumentationContext needs to close events_tx
4. When the Collector sees the channel is closed it needs to call shutdown on the adapter ( https://github.com/dylibso/observe-sdk/blob/main/rust/src/adapter/mod.rs#LL30C37-L30C48 )

For integration with existing traces, we should figure out how to allow the caller to provide a trace ID that can be used instead of generating one. This is primarily for the go otel adapter, but a more generic solution would be nice.

• Change Observe API import module name to be compatible with component model
• Change wasm-instr import module name to be compatible with component model
• Updated Versioning #143
• Error out when observe api is used in go and js sdks as it isn't implemented yet
• Manually readd Go SDK changes from https://github.com/dylibso/observe-sdk/tree/chris/20231003-hack-in-component-support-with-go
• Observe API component support
• Rust-SDK component support

PR: #128

• Remove checks on wasm-instr globals
  • Rust
  • Go
• Support pre-namespace change instr api with deprecation warning
  • Rust
  • Go
  • JS
• Error out pre-linking when observe api with old namespace is used
  • Rust
  • Go
  • JS
    [ ] Add advisory check on version in wasm-instr producers section, Moved out to own issue (#146) as it isn't essential for updated versioning/component support.
• Document api versioning plan so changes can be implemented smoothly.
  -[ ] Glibc is the gold standard on c api versioning https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-library-handles-backward-compatibility#change_your_abi_with_compatibility . See what works with wasm. Linking model is different decided doing it all by hand is easiest for now.

Similar to the Wasm Instr API we should version the observe api so we can detect if a module is compatible with the observe sdk before linking.

Unfortunately adding exported globals for major and minor version numbers may not be doable from source in guest languages as they don't really have an in-language equivalent. We could create a tool to append version numbers to a Wasm module, but we should consider alternative methods that don't require custom tooling.

We need to add an eventbucket for trace events in the rust sdk. Similar to how we do this in the Go SDK: https://github.com/dylibso/observe-sdk/blob/main/go/bucket.go

we should use this in datadog first.

Related to #120

Now that the SDK is not tied to the Rust OTel tracers, we should be able to convert the SDK to use std::time::Instant without issue.

• Go SDK
• Rust SDK
• JS SDK

The default should be applied similarly to each SDK, but have a method for configuring this at the SDK level.

For example, a possible approach is:

let mut trace_ctx = adapter.start(&mut linker, &data)?;
trace_ctx.set_trace_depth(4);

// OR.. avoiding mutability and any additional method surface area:

let config = TraceConfig {
   trace_depth: 4,
   ..default::Default(),
};
let trace_ctx = adapter.start(&mut linker, &data, &config)?;

As discussed over a mail thread, if someone wants to monitor their wasmplugins running inside Envoy proxy which uses Wasmtime runtime, atm observe-sdk can't be used because the runtime is the default one and we can't modify it to have the observe-sdk embedded into it.
Similar scenario happens when we use spin, since spin also uses wasmtime as wasm runtime.

One thing we can do that came out of a discussion with @nilslice is that, we can integrate observe-sdk in wasmtime and create/maintain a fork of wasmtime with builtin observe-sdk.

This issue is to track the discussion/work around this.

We should have some tests. Probably a few unit tests for the adapters and something similar to the rust integration tests.

Running the stdout adapter on ../test/nested.instr.wasm can yield varying results. No triage has been done to determine if it occurs with the other adapters.

One run:

gavin@teleporter:~/dev/observe-sdk/go$ go run bin/stdout/main.go ../test/nested.c.instr.wasm
Hello from Wasm!
2023/11/17 14:59:47  Call to _start took 279.05µs
2023/11/17 14:59:47    Call to __main_void took 227.212µs
2023/11/17 14:59:47      Call to malloc took 20.93µs
2023/11/17 14:59:47      Call to main took 149.579µs
2023/11/17 14:59:47        Call to one took 144.082µs
2023/11/17 14:59:47          Call to two took 138.563µs
2023/11/17 14:59:47            Call to three took 132.061µs
2023/11/17 14:59:47              Call to printf took 125.978µs
2023/11/17 14:59:47                Call to vfprintf took 119.754µs
2023/11/17 14:59:47                  Call to printf_core took 94.882µs
2023/11/17 14:59:47                    Call to __fwritex took 87.077µs
2023/11/17 14:59:47                      Call to __stdio_write took 39.71µs
2023/11/17 14:59:47                        Call to writev took 33.156µs
2023/11/17 14:59:47                          Call to __wasi_fd_write took 21.692µs
2023/11/17 14:59:47    Call to __wasm_call_dtors took 21.599µs

Another run:

gavin@teleporter:~/dev/observe-sdk/go$ go run bin/stdout/main.go ../test/nested.c.instr.wasm
Hello from Wasm!
2023/11/17 14:59:50  Call to _start took 268.672µs
2023/11/17 14:59:50    Call to __main_void took 225.266µs
2023/11/17 14:59:50      Call to main took 158.609µs
2023/11/17 14:59:50        Call to one took 153.519µs
2023/11/17 14:59:50          Call to two took 146.536µs
2023/11/17 14:59:50            Call to three took 138.794µs
2023/11/17 14:59:50              Call to printf took 133.646µs
2023/11/17 14:59:50                Call to vfprintf took 127.749µs
2023/11/17 14:59:50                  Call to printf_core took 105.449µs
2023/11/17 14:59:50                    Call to __fwritex took 98.354µs
2023/11/17 14:59:50                      Call to __stdio_write took 48.64µs
2023/11/17 14:59:50                        Call to writev took 41.878µs
2023/11/17 14:59:50                          Call to __wasi_fd_write took 32.052µs
2023/11/17 14:59:50                            Call to __imported_wasi_snapshot_preview1_fd_write took 26.243µs`

In the second trace __imported_wasi_snapshot_preview1_fd_write shows up which wasn't in the first. However, then __wasm_call_dtors doesn't show up which was in the first trace.

wasmtime, serde, etc could all probably use less strict versions to increase compatibility across variety of end-user codebases. We should look at the "oldest" versions of these that support our needs and make those then minimum requirements. serde being set to "1" is probably safe.

Just as we have a re-usable OTel JSON formatter in Rust, we should support it in Go. This will act as the basis for other adapters to wrap the formatter.

• Go SDK (#124)
• JavaScript SDK
• Rust SDK

After #67

Port the observe API changes for metrics, logs, and manual spans to Go and JS SDKs #51

Related to #48, when we encounter situations that don't really make sense we should have a pattern in place to log some information and if in some operational mode for debug/dev then potentially panic/blow up in a way that informs us or end users of an issue.

This should be consistently disabled by default as to never block user code from executing even if the telemetry is impacted.

Hi, I'd need the required API keys for testing https://github.com/dylibso/observe-sdk#instrumenting-wasm-module
I've reached out over email to get the keys but there's no response yet.

Any help would be greatly appreciated, thanks!

We need repeatable benchmarks around the latency this introduces into executing the module. The benchmarks should focus on initially on anything that's run in line with the module as that's the most important place not to introduce latency. After we are confident in that code path we can concentrate on the collector and the adapters.

• JS
• Go
• Rust

Refactor Go library to match the Rust concurrency architecture: #37

• We should have a generic collector struct / func per module
• it should capture the events
• when it gets a shutdown it should send a new object, a tracevent payload, to the singleton adapter func

Working on Updated Versioning. I added deprecation warnings for the old namespace dylibso_observe. However, I'm not sure they are going the right place or actually making it to the observe-sdk user across the languages and adapters. We should decide on a consistently policy for this and apply it everywhere.

Currently, the rust sdk outputs with log::warn! and the basic example is setup to by default log with warnings.

The go-sdk outputs to the standard logger with log.Println.

The js-sdk outputs with console.warn.