mikeinnes / diff-zoo Goto Github PK

First I want to thanks you because this is perfectly done!
It help me a lot to understand because your truly start from the basics with very high quality explanation.

-LaTeX rendering bug after out[31] (in the intro part).

-In[20] (in the forward part) could be write with LaTeX in the text to help fast readers.
Also this section "Perturbation Confusion" could have a conclusion about solutions or the lack of solution to fix this problem.

It should be divm (not mulm).

Thank you for this extremely wonderful tutorial. I learnt a lot.
Also please mention that they need to read backandforth.ipynb after the intro.ipynb. It makes most sense that way. And it took me a while to figure that out.

And now the bug...

struct Dual{T<:Real} <: Real
    x::T
    y::T
end
Base.show(io::IO, d::Dual) = println(io, d.x, d.y≥0 ? "+" : "", d.y, "ϵ")

import Base: +, -, *, /
a::Dual + b::Dual = Dual(a.x + b.x, a.y + b.y)
a::Dual * b::Dual = Dual(a.x * b.x, b.x * a.y + a.x * b.y)
a::Dual / b::Dual = Dual(a.x / b.x,(b.x * a.y - a.x * b.y)/b.x^2)

Base.convert(::Type{Dual{T}}, a::Dual) where T = Dual(convert(T, a.x), convert(T, a.y))
Base.convert(::Type{Dual{T}}, a::Real) where T = Dual(convert(T, a), zero(T))
Base.promote_rule(::Type{Dual{T}}, ::Type{R}) where {T,R} = Dual{promote_type(T,R)}

Now let us try a simple function. I am also giving a numerical derivative for reference.

f(x) = x/(1+x*x)
ndf(x, ϵ=√eps()) = (f(x+ϵ)-f(x-ϵ))/2ϵ
f(7), ndf(7)

(0.14, -0.01919999998062849)

Now let us try with Dual numbers

f(Dual(7., 1.))

0.14-0.0192ϵ

Works!

But with your definition of division,

a::Dual / b::Dual = Dual(a.x * b.x, b.x * a.y - a.x * b.y)
f(Dual(7., 1.))

350.0-48.0ϵ

It is obviously wrong. Also you can see that my formula for division matches with your high school text book.

Just a nudge to please add the source-to-source reverse mode notebook.

The first sentence reads:

These notebooks are an exploration of various approaches to analytical differentiation.

From what I can tell, "analytical" should be "algorithmic".

Hi , awesome tutorial !!! Hope can finish reverse notebook, very interesting in theory behind Zygote.jl

┌ Error: error when executing notebook based on input file: `<path>\diff-zoo\src\tracing.jl`
└ @ Literate <path>\.julia\packages\Literate\3qT7A\src\Literate.jl:555
ERROR: LoadError: LoadError: syntax: unexpected semicolon in tuple around string:1
in expression starting at string:1
when executing the following code block

@eval dcube(x) = $(Expr(dy))

dcube(5)

If I comment out these lines, then the next lines error too.
I tested with Julia 1.6 and 1.10 and with Literate v2.2.0 and v2.16.0. All had the same error.
There is also a Pkg error updating dependency versions unless name = "diff-zoo" is removed from the Project.toml.