Giter VIP home page Giter VIP logo

Comments (3)

mcabbott avatar mcabbott commented on May 25, 2024 1

Ok, that sounds like a pretty thorny issue. Glad to hear it's not my fault though!

from tullio.jl.

mcabbott avatar mcabbott commented on May 25, 2024

That isn't good. Here's what I see (on a 2-core machine, Julia 1.5.2, everything updated):

julia> @btime Zygote.gradient(f, x)[1];
  9.570 ms (163 allocations: 30.53 MiB)

julia> @btime Zygote.gradient(ft, x)[1];
  13.836 ms (26 allocations: 30.52 MiB)

# forward only
julia> @btime f($x);
  949.564 μs (92 allocations: 6.03 KiB)

julia> @btime ft($x);
  1.825 ms (0 allocations: 0 bytes)

I wonder what's causing it to be so different? Do you see a slowdown on the forward-only evaluation too?

from tullio.jl.

roflmaostc avatar roflmaostc commented on May 25, 2024

Hey,

I've got the same versions on my machine. I'm not sure what exactly happens, but in a fresh REPL I get similar results.

using Zygote, Tullio, BenchmarkTools
x = abs.(randn((500, 500)));
f(x) = (@tullio y =  abs2(x[i+1, j] - x[i-1, j]) + abs2(x[i, j+1] - x[i, j-1]))
ft(x) = (@tullio threads=false y = abs2(x[i+1, j] - x[i-1, j]) + abs2(x[i, j+1] - x[i, j-1]))

@btime f($x);
@btime ft($x); 

@btime Zygote.gradient($f, $x)[1];
@btime Zygote.gradient($ft, $x)[1];

returns

  40.286 μs (92 allocations: 6.36 KiB)
  77.808 μs (0 allocations: 0 bytes)
  574.786 μs (162 allocations: 1.92 MiB)
  783.779 μs (11 allocations: 1.91 MiB)

However, initially I tested this in a larger Jupyter notebook where additional packages were loaded. After testing each package separatly found the source:

using Zygote, Tullio, BenchmarkTools, ImageView
x = abs.(randn((500, 500)));
f(x) = (@tullio y =  abs2(x[i+1, j] - x[i-1, j]) + abs2(x[i, j+1] - x[i, j-1]))
ft(x) = (@tullio threads=false y = abs2(x[i+1, j] - x[i-1, j]) + abs2(x[i, j+1] - x[i, j-1]))

@btime f($x);
@btime ft($x); 

@btime Zygote.gradient($f, $x)[1];
@btime Zygote.gradient($ft, $x)[1];

returns

  8.161 ms (93 allocations: 6.38 KiB)
  74.629 μs (0 allocations: 0 bytes)
  13.679 ms (164 allocations: 1.92 MiB)
  750.308 μs (11 allocations: 1.91 MiB)

Note that for threaded we see ms and not µs.

Profile (with a for loop to increase total computing time) inspection suggests that some GTK functions are involved.

Without Threads:


106 (35 %) | nothing
-- | --
80 (26 %) | /usr/bin/../share/julia/base/./fastmath.jl
70 (23 %) | /usr/bin/../share/julia/base/./array.jl
42 (14 %) | /home/fxw/.julia/packages/IJulia/rWZ9e/src/execute_request.jl

With Threads:


1002 (70 %) | /home/fxw/.julia/packages/Gtk/C22jV/src/events.jl
-- | --
144 (10 %) | nothing
90 (6 %) | /usr/bin/../share/julia/base/./fastmath.jl
85 (6 %) | /usr/bin/../share/julia/base/./array.jl
52 (4 %) | /home/fxw/.julia/packages/IJulia/rWZ9e/src/execute_request.jl

Searching for GTK performance issues brings me to: JuliaGraphics/Gtk.jl#503, JuliaLang/julia#35552

So I'm sorry that I posted it here since it turns out to be nothing caused by Tullio.
But still, that's imo a disappointing issue.

Felix

from tullio.jl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.