Giter VIP home page Giter VIP logo

Comments (6)

mohamed82008 avatar mohamed82008 commented on June 12, 2024 1

When you time with @btime use $, e.g. @btime Bijectors.forward($cb, $x)

from bijectors.jl.

xukai92 avatar xukai92 commented on June 12, 2024 1

Looks great to me.

BTW can someone explain a bit on the benchmark time difference with and without $?

from bijectors.jl.

torfjelde avatar torfjelde commented on June 12, 2024 1

Also, we'll still keep the "simple" iterative implementation. So we have two different ways of constructing a Composed; using a Tuple or Array. Using Tuple gives type-stable implementations using @generated, using Array gives unstable iterative implementations.

from bijectors.jl.

willtebbutt avatar willtebbutt commented on June 12, 2024

Nice work @torfjelde . Could you provide some benchmarks for vector-valued transformations? I would imagine that these are where we're most likely to see deeply nested structures, so it would be interesting to know if the performance gains persist here or not.

from bijectors.jl.

torfjelde avatar torfjelde commented on June 12, 2024

It does but actual transformation-cost of course reduces the difference:

julia> using Bijectors: PlanarLayer

julia> b = PlanarLayer(10);

julia> x = randn(10);

julia> @btime Bijectors.forward($cb, $x)
  2.912 μs (47 allocations: 4.63 KiB)
(rv = [-0.226818, -5.05146, -1.45391, 2.3923, 2.71439, 1.76895, -0.988026, 1.12392, 3.00605, -2.77136], logabsdetjac = -0.019665352458356412)

julia> @btime Bijectors.forward_gen($cb, $x)
  2.922 μs (47 allocations: 4.63 KiB)
(rv = [-0.226818, -5.05146, -1.45391, 2.3923, 2.71439, 1.76895, -0.988026, 1.12392, 3.00605, -2.77136], logabsdetjac = -0.019665352458356412)

julia> @btime Bijectors.forward_it($cb, $x)
  3.305 μs (49 allocations: 4.73 KiB)
(rv = [-0.226818, -5.05146, -1.45391, 2.3923, 2.71439, 1.76895, -0.988026, 1.12392, 3.00605, -2.77136], logabsdetjac = -0.019665352458356412)

and for deep ones where recursion is no longer inlined:

julia> cb = foldl(, [b for i = 1:50]);

julia> @btime Bijectors.forward($cb, $x)
  116.340 μs (1299 allocations: 128.05 KiB)
(rv = [0.56373, -8.38962, -2.83878, 3.42109, 4.44716, 2.13511, -0.910896, 2.02499, 4.82258, -3.82649], logabsdetjac = -168.86605036964167)

julia> @btime Bijectors.forward_gen($cb, $x)
  70.403 μs (1151 allocations: 114.88 KiB)
(rv = [0.56373, -8.38962, -2.83878, 3.42109, 4.44716, 2.13511, -0.910896, 2.02499, 4.82258, -3.82649], logabsdetjac = -168.86605036964167)

julia> @btime Bijectors.forward_it($cb, $x)
  75.723 μs (1153 allocations: 115.84 KiB)
(rv = [0.56373, -8.38962, -2.83878, 3.42109, 4.44716, 2.13511, -0.910896, 2.02499, 4.82258, -3.82649], logabsdetjac = -168.86605036964167)

A further test, also including an extreme case of Stacked from #36 (which also uses @generated for forward to get type-stability)

julia> using Bijectors, BenchmarkTools

julia> using Bijectors: Logit

julia> D = 100
100

julia> sb = Stacked(tuple([Logit(0.0, 1.0) for i = 1:D]...));

julia> bs = [
           ("PlanarLayer($D)", PlanarLayer(D)),
           ("RadialLayer($D)", RadialLayer(D)),
           ("Stacked{Logit ∘ inv(Logit), $D}", sb  inv(sb))
       ];

julia> x = randn(D);

julia> for (name, b) in bs
           @info "$(name): b ∘ b"
           cb = b  b
           @btime Bijectors.forward($cb, $x)
           @btime Bijectors.forward_gen($cb, $x)
           @btime Bijectors.forward_it($cb, $x)

           N = 50
           @info "$(name): b ∘ b ∘ ... ∘ b ($N times)"
           cb = foldl(, [b for i = 1:N])
           @btime Bijectors.forward($cb, $x)
           @btime Bijectors.forward_gen($cb, $x)
           @btime Bijectors.forward_it($cb, $x)
       end
[ Info: PlanarLayer(100): b  b
  4.746 μs (47 allocations: 14.69 KiB)
  4.758 μs (47 allocations: 14.69 KiB)
  5.091 μs (49 allocations: 14.80 KiB)
[ Info: PlanarLayer(100): b  b  ...  b (50 times)
  162.648 μs (1299 allocations: 379.61 KiB)
  116.781 μs (1151 allocations: 366.44 KiB)
  119.156 μs (1153 allocations: 367.41 KiB)
[ Info: RadialLayer(100): b  b
  3.263 μs (35 allocations: 8.53 KiB)
  3.287 μs (35 allocations: 8.53 KiB)
  3.642 μs (37 allocations: 8.64 KiB)
[ Info: RadialLayer(100): b  b  ...  b (50 times)
  131.128 μs (999 allocations: 225.70 KiB)
  82.406 μs (851 allocations: 212.53 KiB)
  85.703 μs (853 allocations: 213.50 KiB)
[ Info: Stacked{Logit  inv(Logit), 100}: b  b
  69.882 μs (2009 allocations: 194.28 KiB)
  70.363 μs (2009 allocations: 194.28 KiB)
  82.146 μs (2028 allocations: 260.78 KiB)
[ Info: Stacked{Logit  inv(Logit), 100}: b  b  ...  b (50 times)
  9.063 ms (55549 allocations: 35.88 MiB)
  1.830 ms (50201 allocations: 4.74 MiB)
  7.016 ms (50700 allocations: 35.88 MiB)

from bijectors.jl.

torfjelde avatar torfjelde commented on June 12, 2024

Closed by #52

from bijectors.jl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.