Array `getindex` rule unable to handle Zero types and `NotImplemented` about chainrules.jl HOT 7 OPEN

Oh it is arrays containing Zeros/NotImplemented?
Yeah we should probably handle that.
Though if it only contains Zero at least, it should really be being simplified down to just Zero somewhere. (not sure where)

from chainrules.jl.

I will try to get back to this and the PR which spawned it this weekend. IIRC doing some types of collapsing made certain Zygote tests very unhappy.

from chainrules.jl.

Maybe the correct solution is to catch this at a higher level.

Yes, this should be handled in the AD system before it calls pullback.
Because if the input to pullback is one of these types we know the output (it will be the same object) so we don't need to handle it in the rules.

from chainrules.jl.

If I'm not mistaken it'd have to be handled in ∇getindex? That's what I meant by higher level. Otherwise the AD system would have to know a priori that this particular pullback will blow up when it encounters a NotImplemented/is a no-op when it's passed an array of AbstractZero.

from chainrules.jl.

Collapsing arrays of Zeros seems reasonable. Is there a rule for what happens when that array contains both ZeroTangent and NoTangent? Would simplify_cotangents(x::Array{<:AbstractZero}) = ZeroTangent() (or = NoTangent()) be sufficient?

from chainrules.jl.

Is there a rule for what happens when that array contains both ZeroTangent and NoTangent?

Technically speaking we should keep them seperate.
In practice seeing a mix should be really rare.
And noone actually treats them differently.
Simplifying them all to NoTangent() is probably fine.
(NotIplemented dominates NoTangent dominates ZeroTangent)

from chainrules.jl.

I can't make a MWE for this, that isn't purely zeros (which should be removed before hitting the rrule)

E.g. following works

_, back3 = rrule(getindex, [10, 0, -1], :)
gs3 = back3([2.0, NoTangent(), (@not_implemented "test2")])
num, notan, not_imp = unthunk(gs3[2]) 
@test  num isa Real
@test iszero(notan)  # We don't care if this gets converted to a 0.0
@test not_imp isa NotImplemented

from chainrules.jl.