adapted_grid, why not use end points of interval about plotutils.jl HOT 27 CLOSED

@mkborregaard I don't think the log scale is the cause of the issue here (though it would make sense to have a curvature criteria depend on the scales of the axes).

from plotutils.jl.

I don't think I saw this but this is plotted on a log scale. I think that is a quite important thing to take into account which the current code doesn't. But I see now it has been mentioned before...

from plotutils.jl.

Good point - I'll tag @KristofferC who should know

from plotutils.jl.

There was some good reason for it, but I don't really recall what. Maybe that the function is usually badly behaved at the exakt endpoints (think 1/x between 0 and 1). If this significantly changes the look of the plot then you probably need to have a smaller interval.

from plotutils.jl.

Well, I think the whole point of this function is that you specify a domain over which you want the plot, and it figures out the appropriate interval to make the plot look nice.

I can work on a PR at some point, but I've played around with small modifications on the adapted_grid function:
goretkin/finite_diff@830eeca...master

Here's the difference:

Moving away from the endpoints, in the current implementation, completely misses the inflection point. To adapted_grid, the whole graph looks linear. This happens even if you ask for more refinement:

(plotting code in the repo, but reproduced here for posterity)

using Plots

f(x) = x^2
∇f(x) = 2x

forward_difference(f, x, h) = (f(x+h) - f(x)) / h

signed_error(f, ▽f, x, h, scheme) = scheme(f, x, h) - ▽f(x)

error_plot = plot(;xaxis=:log, yaxis=:log)

plotme(h) = abs(signed_error(f, ∇f, 1, h, forward_difference))

(h_start, h_end) = (1e-20, 0.1)

plot!(error_plot, plotme, h_start, h_end
  ;
  label="current adapted_grid",
  marker=:x,
  markersize=1.0,
  linewidth=4.0
)

include("adapted_grid.jl")
max_recursions = 7
h_eval = gg_adapted_grid(plotme, (h_start, h_end); max_recursions=max_recursions)
y_eval = plotme.(h_eval)
#y_eval[y_eval .== 0] .= 1e-20 # unrelated Plots issue: https://github.com/JuliaPlots/Plots.jl/issues/2046

plot!(error_plot, h_eval, y_eval,
  ;
  label="gg_adapted_grid, max_recursions: $(max_recursions)",
  marker=:o,
  markersize=0.1,
)

from plotutils.jl.

Does anyone remember which repo this code was moved from? It might be beneficial for me to read the discussion. It seems that since then, adapted_grid handles infinity values with no problem, and so does Plots:

f(x) = 1/x
xs = gg_adapted_grid(f, (0.0, 1.0))
plot(xs, f.(xs); marker=:o)

julia> xs[1]
0.0

from plotutils.jl.

I think Kristoffer wrote the code for this repo.
@goretkin how much of your issue do you think is due to the adapted_grid function not realizing that the graph is going to be plotted on a log scale? That seems like an obvious thing to fix first.

from plotutils.jl.

Why not - is the space moved in from the given endpoints not measured in arithmetic terms, thus blowing it up greatly on a log scale?

from plotutils.jl.

Yes, however that wouldn't be a problem if we didn't move in from the endpoints ;)

Here's the same data on a linear-linear plot

Even in this space, adapted_grid misses the important feature of the function.

from plotutils.jl.

Yes I see - you could try to open a PR that does not move in the end points, and then consider having a graceful way of resolving the situation (like adaptively moving in) if the function is poorly behaved at the end point (a logistic progression on (0,1) being a good example)?

from plotutils.jl.

Wait, I'm confused. It only moves the interior points slightly. Not the endpoints?

The comment I had earlier is probably me misremembereming, and a red herring. I probably did consider infs when making this algorithm.

from plotutils.jl.

@KristofferC True, it does perturb only interior points. But it also discards the endpoints. From the docstring:
Functions are never evaluated exactly at the end points of the intervals.

See goretkin/finite_diff@830eeca...master for exact places where the code avoids evaluating at the endpoints

from plotutils.jl.

Here is the original discussion JuliaPlots/Plots.jl#621

from plotutils.jl.

See eg JuliaPlots/Plots.jl#621 (comment).

from plotutils.jl.

Thanks for finding that! Do my examples above convince you that evaluating at the endpoints is probably okay or perhaps necessary? I don't think there's any scale-invariant way to move away from the endpoints.

from plotutils.jl.

Since mathematica doesn't do that I don't think it is necessary. Perhaps we need more initial points.

from plotutils.jl.

No, I don't think more initial points would fix it.

I am not sure why mathematica doesn't evaluate the function at the endpoints, but my current thinking is that it's a hack. And I do not think it is a necessary hack that we need to inherit.

We currently have the hack in place because otherwise:

julia> f(x) = sin(1/x)
f (generic function with 1 method)

julia> xs = gg_adapted_grid(f, (0.0, 1.0));
ERROR: DomainError with Inf:
sin(x) is only defined for finite x.

But this is a problem that comes up pretty often. https://github.com/mlubin/NaNMath.jl is designed to fix it.

julia> fnan(x) = NaNMath.sin(1/x)
fnan (generic function with 1 method)

julia> xs = gg_adapted_grid(fnan, (0.0, 1.0));

julia> Plots.plot(xs, fnan.(xs))

Instead of throwing a domain error, NaNMath.sin returns NaN. Plots can already handle NaNs beautifully.

It is annoying, though, to get DomainError inside of a plotting function. So I think instead we should swallow it up. Only if we get a DomainError should we start ignoring points. In fact this would be a more robust solution than is currently in place, since domain errors can happen anywhere, not just at endpoints.

julia> f_bad(x) = sin(1/((x-0.5)^100))
f_bad (generic function with 1 method)

julia> f_bad(0.0)
-0.8721836054182673

julia> f_bad(1.0)
-0.8721836054182673

julia> f_bad(0.5001)
ERROR: DomainError with Inf:
sin(x) is only defined for finite x.
Stacktrace:
 [1] sin_domain_error(::Float64) at ./special/trig.jl:28
 [2] sin(::Float64) at ./special/trig.jl:39
 [3] f_bad(::Float64) at ./REPL[55]:1
 [4] top-level scope at none:0

Indeed:

julia> Plots.plot(f_bad, 0, 1)
ERROR: DomainError with Inf:
sin(x) is only defined for finite x.

from plotutils.jl.

I think some screenshots from plotting this function in other software would give some more information.

However, I am no too attached with the exact implementation omwe have now. Changes to it needs to be accompanied with thorough tests and comparisons with the previous implementation of different types of functions though.

from plotutils.jl.

I agree with @KristofferC that we should be open to change the behaviour here, but that it would require quite extensive testing of weird functions and edge cases. If you're up for it @goretkin ? There's a link to some "horror functions" that would make for good tests in the Plots issue that Kristoffer linked above.

from plotutils.jl.

@goretkin what do you suggest here?

from plotutils.jl.

The log scale isn't the problem here (though assuming the data will be plot on a linear scale is not great either). It's the handling of the endpoints.

My suggestion is #63 (comment)

I started breaking out this functionality, so that I could change it to evaluate the function at the endpoints without breaking any existing behavior: https://github.com/goretkin/PlotAdaptiveGrid1D.jl but I never finished it up.

from plotutils.jl.

I got hung up on making a testing infrastructure for this. I wanted something that would test for features of the true function being represented in the sampled reconstruction, like local extrema and zero crossings.

from plotutils.jl.

oh wow. Maybe just make a small improvement with some tests and chunk it up a bit?

from plotutils.jl.

Why wouldn't taking into account log scale be important? We want to plot to look nice but the curvatures we compute are only slightly related to the curvature that ends up on the plot.

from plotutils.jl.

No I agree that is important.
My suggestion was simply not to get caught up in a super-solid mega-PR, when less would already be an improvement.

from plotutils.jl.

Maybe this way of drawing a pitchfork-bifurcation is a good test case, that illustrates the shortcomings of the current implementation (the gap around the origin):

julia> using Plots

julia> plot(x->sqrt(x),0,10)

julia> plot!(x->-sqrt(x),0,10)

julia> plot!(x->0,-10,0)

julia> plot!(x->0,0,10, linestyle=:dash)

from plotutils.jl.

We now use endpoints.
Technically one could avoid evaluation at the endpoints, but one should only if the result is optically indistinguishable.

from plotutils.jl.