[Bug] constructor for `OptionsNuisance` modifies its own arguments about hiqga.jl HOT 5 CLOSED

I am about to submit a PR on this, but I've also noted a couple of oddities in the nuisance sampling code that should probably be fixed in the future:

1. Every time a nuisance move is made an entirely new copy of the associated options structure is made. This is wasteful and will slow inversions down, and it appears the only reason it is done is to allow rotating the sampling space according to some supplied covariance matrix.
2. Even if a covariance matrix is not supplied, the code is still calculating "rotated" versions of the prior bounds and using those instead. This reduces the numerical accuracy of the bounds that the sampler is actually using and performs a bunch of unnecessary matrix diagonalisation/multiplication.

from hiqga.jl.

We could fix the diagonalisation yes but more than 3 or 4 nuisances aren't feasible to invert without diagonalisation anyway ... for these the unnecessary diagonalisation is pretty fast. Agreed there should be a better way, I just didn't want to write a new oneaxis move and reused the old one, but it needs an optn with rotated bounds.

from hiqga.jl.

Yeah I think the linear algebra ops are fairly cheap for a small number of nuisance parameters.

The comment about not being able to sample without rotating to principal components for more than a few nuisance parameters surprises me - MCMC is supposed to be good at producing samples from multi-dimensional distributions, and I did a simple test using this script to compare Metropolis-Hastings sampling efficiency of a highly-correlated normal distribution with and without rotating the sampling coordinates. Qualitatively, it looks about the same to me:

Does this mean that if we had 3 or 4 uncorrelated nuisance parameters we would be completely unable to sample them properly?

from hiqga.jl.

In any case, once we're ready to progress changes to the nuisance sampler further it's probably best to open a separate issue referencing this one.

from hiqga.jl.

No actually, if there are strong correlations ... unrotated McMC propsals walk on coordinate axes which are not aligned with the correlation directions. This is extremely inefficient. While not apparent in 2 dimensions, 3 or 4 highly correlated parameters are ok, more are problematic. If you use a sampled correlation for say three Tempest nuisances, it is much quicker to converge. That's why the initial synthetics had so much trouble ... but since the forwards were cheap we said may as well sample longer instead of using a sample covariance. See this for a more detailed explanation in the proposal part.

from hiqga.jl.