Discrepancies between model parameters and periodogram about fasttemplateperiodogram HOT 6 CLOSED

Interesting – thanks for putting this together.

Given how complex the various routines are that lead to this, I think the most fruitful way to track down the bug will be to make sure each component is well tested. My suspicion from playing around with the nfft is that there may be corner cases where the fast summations do not accuratly approximate the true summations – if we could add an option to build the periodogram using direct summations, that could help.

It might also be some kind of interaction between errors in root finding and errors in summation approximation. But I wouldn't dive into those kind of second order effects until each component is understood and validated by comprehensive unit tests.

from fasttemplateperiodogram.

Hi Jake,

Thanks for the feedback -- the computations above (which use the new fit_template function) only use direct_summations, so this is independent of the NFFT. I suspect there may be some numerical instabilities somewhere that cause the problem at certain phase shifts. Root finding from computed coefficients is known to be unstable with certain algorithms and with certain classes of polynomials (closely spaced roots, large variations in coefficient amplitudes, etc), so that may explain some of the problem.

However, I'm still currently baffled by why the reported fit parameters should be inconsistent with the reported periodogram value, since they both are computed from the same set of variables, and the periodogram computation depends explicitly on the phase shift.

I'll keep adding tests and digging deeper. I'm working on reviewing a JOSS article at the moment, so I'll get back to this a bit later tonight, but feel free to poke around if you have any ideas.

John

from fasttemplateperiodogram.

Another note -- this branch implements unit tests that compare direct_summations against fast_summations, and everything passes. Doesn't have any bearing on this problem (I don't think), but that should give some additional peace of mind about the NFFT operations. And, since they're unit tested, you can run amok with any speedup ideas for the fast_summations function if you feel like it.

from fasttemplateperiodogram.

Ah, I missed that part. Good to know.

from fasttemplateperiodogram.

Long story short, I tracked this down to a couple of issues.

• One was a mistake in the PseudoPolynomial.root_finding_poly() method. Originally, the method would return (1 - x^2)p^2(x) - q^2(x), however, this should be p^2(x) - (1 - x^2)q^2(x).

• The phase shift parameter, which has caused much grief, was also incorrect, and this was a result of the affine transformation of the observation times to the (-1/2, 1/2) range that was required by NFFT. I had hacked together what I thought was a fix, but this should really be taken care of by the fast_summations function. Now, the results of the NFFT are multiplied by a set of twiddle factors, which correct for the transformation.

Everything should now be fixed as of commit b30e775ef1684c64d3e5574dce30bd7ad8d6241a.

from fasttemplateperiodogram.

Great! Glad to hear you were able to track-down these problems.

from fasttemplateperiodogram.