Giter VIP home page Giter VIP logo

ocaml-trace's Introduction

Trace

Build and Test

This small library provides basic types that can be used to instrument a library or application, either by hand or via a ppx.

Features

  • spans
  • messages
  • counters
  • other metrics?
  • ppx to help instrumentation

Usage

To instrument your code, you can simply add trace to your dune/opam files, and then write code like such:

let f x =
  Trace.with_span ~__FILE__ ~__LINE__ "inside-f" @@ fun _sp ->
  (* … code for f *)

let g x =
  Trace.with_span ~__FILE__ ~__LINE__ "inside-g" @@ fun _sp ->
  let y = f x in
  (* … code for f *)

let () =
  Some_trace_backend.setup () @@ fun () ->
  let result = g 42 in
  print_result result

The file test/t1.ml follows this pattern, using trace-tef as a simple backend that emits one JSON object per span/message:

let run () =
  Trace.set_process_name "main";
  Trace.set_thread_name "t1";

  let n = ref 0 in

  for _i = 1 to 50 do
    Trace.with_span ~__FILE__ ~__LINE__ "outer.loop" @@ fun _sp ->
    for _j = 2 to 5 do
      incr n;
      Trace.with_span ~__FILE__ ~__LINE__ "inner.loop" @@ fun _sp ->
      Trace.messagef (fun k -> k "hello %d %d" _i _j);
      Trace.message "world";
      Trace.counter_int "n" !n
    done
  done

let () =
  Trace_tef.with_setup ~out:(`File "trace.json") () @@ fun () ->
  run ()

After running this, the file "trace.json" will contain something like:

[{"pid":2,"name":"process_name","ph":"M","args": {"name":"main"}},
{"pid":2,"tid": 3,"name":"thread_name","ph":"M","args": {"name":"t1"}},
{"pid":2,"cat":"","tid": 3,"ts": 2.00,"name":"hello 1 2","ph":"I"},
{"pid":2,"cat":"","tid": 3,"ts": 3.00,"name":"world","ph":"I"},
{"pid":2,"tid":3,"ts":4.00,"name":"c","ph":"C","args": {"n":1}},

Opening it in https://ui.perfetto.dev we get something like this:

screenshot of perfetto UI

ppx_trace

On OCaml >= 4.12, and with ppxlib installed, you can install ppx_trace. This is a preprocessor that will rewrite like so:

let%trace f x y z =
  do_sth x;
  do_sth y;
  begin
    let%trace () = "sub-span" in
    do_sth z
  end

This more or less corresponds to:

let f x y z =
  let _trace_span = Trace_core.enter_span ~__FILE__ ~__LINE__ "Foo.f" in
  match
    do_sth x;
    do_sth y;
    begin
      let _trace_span = Trace_core.enter_span ~__FILE__ ~__LINE__ "sub-span" in
      match do_sth z with
      | res ->
        Trace_core.exit_span _trace_span;
        res
      | exception e ->
        Trace_core.exit_span _trace_span
        raise e
    end;
  with
  | res ->
    Trace_core.exit_span _trace_span
    res
  | exception e ->
    Trace_core.exit_span _trace_span
    raise e

Alternatively, a name can be provided for the span, which is useful if you want to access it and use functions like Trace.add_data_to_span:

let%trace f x y z =
  do_sth x;
  do_sth y;
  begin
    let%trace _sp = "sub-span" in
    do_sth z;
    Trace.add_data_to_span _sp ["x", `Int 42]
  end

Dune configuration

In your library or executable stanza, add: (preprocess (pps ppx_trace)). The dependency on trace.core is automatically added. You still need to configure a backend to actually do collection.

Backends

Concrete tracing or observability formats such as:

ocaml-trace's People

Contributors

c-cube avatar joprice avatar tatchi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

ocaml-trace's Issues

way to enter an implicit context from explicit data

say you receive via HTTP a trace-id/span ID and want to process that request synchronously using the implicit with_span interface. There needs to be a way to associate the incoming contextual information with the current scope, without necessarily entering a fresh new span from it.

Something like

val with_enter_implicit_context : span -> (unit -> 'a) -> 'a

and then you're inside that span. It works with an explicit span too because these contain a regular span.

cc @ELLIOTTCABLE

Suggestion to use `__FUNCTION__`

I have not yet used the library but assume that most users at least initially would instrument functions by saying:

Trace.with_span ~__FILE__ ~__LINE__ __FUNCTION__ @@ fun _sp ->

Only when you need to trace parts inside a function, you would need a different name. There is also __LOC__ available. I was wondering, why you didn't suggest to use __FUNCTION__? The code above could be easily inserted using search/replace in an editor.

Addendum: I looked at the implementation and indeed it encourages to use __FUNCTION__:

let[@inline] with_span ?__FUNCTION__ ~__FILE__ ~__LINE__ ?data name f =

So you can ignore my comment.

Trace_core.add_data_to_span doesn't work for with_span

I see there's a new release that looks relevant, so this might have been fixed already. As of 0.6, Trace_core.add_data_to_span works if you use enter_span and exit_span but not with_span. So

      let sp = Tracing.enter_span ~__FILE__ ~__LINE__ "Core_CLI.main_no_exn_handler" in
      let res = f () in
      Tracing.add_data_to_span sp ["test", `String "appear please"];
      Tracing.exit_span sp; 
      res

works

But not

  Tracing.with_span ~__FILE__ ~__LINE__ "Core_CLI.main_no_exn_handler"
    (fun sp -> 
              Tracing.add_data_to_span sp ["test", `String "appear please"];
              f ())

Not urgent but would be nice to have since with_span makes child spans.

Using the ppx doesn't seem compatible with the opentelemetry backend

I'm trying to use the ppx for tracing, since it looks really good! However, I'm having trouble using the opentelemetry backend with trace. Installing ppx_trace seems to make opentelemetry.trace no longer present.

Here's the process:

This is how I'm currently using opentelemetry. I have a wrapper with_setup function that I pass the function I want to trace. This has been working great.

let with_setup f =
  let otel_backend =
    Opentelemetry_client_ocurl.create_backend ()
  in
  Opentelemetry_trace.setup_with_otel_backend otel_backend;

  Opentelemetry_client_ocurl.with_setup () @@ fun () ->
  run_with_span "All time" f

When I install ppx_trace, opam tells me these are the necessary conflict resolution actions.

➜  semgrep git:(develop) ✗ opam install ppx_trace
Constructing initial basis...
Number of 0-1 knapsack inequalities = 20710
Constructing conflict graph...
Conflict graph has 3444 + 1337 = 4781 vertices
The following actions will be performed:
  ↘ downgrade opentelemetry                   0.6 to 0.4
  ↗ upgrade   trace                           0.4 to 0.6 [required by ppx_trace]
  ↘ downgrade opentelemetry-client-ocurl      0.6 to 0.4 [uses opentelemetry]
  ↘ downgrade opentelemetry-client-cohttp-lwt 0.6 to 0.4 [uses opentelemetry]
  ∗ install   ppx_trace                       0.6
  ↗ upgrade   trace-tef                       0.4 to 0.6 [uses trace]
===== ∗ 1   ↗ 2   ↘ 3 =====
Do you want to continue? [Y/n] y

When I downgrade opentelemetry and opentelemetry-client-ocurl, it seems that I lose opentelemetry.trace:

➜  semgrep git:(develop) ✗ make core;n
/Library/Developer/CommandLineTools/usr/bin/make minimal-build
dune build _build/install/default/bin/semgrep-core
File "libs/tracing/unix/dune", line 9, characters 3-22:
9 |    opentelemetry.trace
       ^^^^^^^^^^^^^^^^^^^
Error: Library "opentelemetry.trace" not found.
-> required by library "tracing.unix" in _build/default/libs/tracing/unix
-> required by executable Main in src/main/dune:2
-> required by _build/default/src/main/Main.exe
-> required by _build/install/default/bin/semgrep-core
make[1]: *** [minimal-build] Error 1             
make: *** [core] Error 2

This means Opentelemetry_client_ocurl.create_backend and Opentelemetry_trace.setup_with_otel_backend are no longer available.

What's the recommended way to set up opentelemetry and also use the ppx?

Depend on `trace` instead of `trace.core` leads to module resolution failure

Installing trace.0.2 from opam, and adding it to the dune deps:

------ dune 100644
++++++ dune 100644
@|-51,33 +51,35 ============================================================
 |  xmlm
 |  zip
+|  trace
 |  opentelemetry
 |  opentelemetry-client-ocurl)
 | (preprocess
 |  (per_module
 |   ((action
 |     (run camlp4o %{input-file}))

… then referring to Trace.Collector.S, results in a compile-error:

File "trace_collector.ml", line 58, characters 17-34:
58 |   let module M : Trace.Collector.S = struct
                      ^^^^^^^^^^^^^^^^^
Error: The module Trace.Collector is an alias for module Trace_core__.Collector, which is missing

Did something fail to get properly copied/exposed in the v0.2 refactor?

Apologies if it turns out this is something specific to our architecture. 😅

duplicate module definition when running dune utop

I get an error when trying to start utop via dune, where both the toplevel and this library provide a module named "Trace":

Error: Files /nix/store/sqkjv5i2qdwlf2244ifs9ga50m011p05-ocaml5.1.0-beta1-trace-n-a/lib/ocaml/5.1.0~beta1/site-lib/trace/trace.cma(Trace)

       and /nix/store/46vkfb33m24jjl2rj40cd3qfgg8d3inq-ocaml+flambda-5.1.0-beta1/lib/ocaml/compiler-libs/ocamltoplevel.cma(Trace)

       both define a module named Trace```

Extend API to allow the addition of attributes to unclosed-but-created spans

At the moment, the ~data API requires all attributes to be known at span-creation time. This isn't always possible.

Consider the case of an HTTP request, on the client-side — when the span is opened, one cannot know the response-code of that request; but before the span is entirely closed, you'd want to attach that information to the span.

Something like Trace.add_data ~data span_id, perhaps?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.