Validating grains - completeness etc about imaged11 HOT 4 OPEN

makemap.py should now output map files with the number of uniq peaks in from commit f957c28
Still need to put this into indexing (either as merge before indexing or checking afterwards).

from imaged11.

Note from #81": To deal better with overlaps, and try to overcome a few recurring problems:

• Can we convert columnfile.py into a pandas dataframe ?
• Raw peak data in one table (x,y,omega,intensity etc)
• Detector geometry applied -> XL, YL, ZL : adds a new table depending on geometry
• Depending on (UB)+Diffractometer -> OmegaCalc : adds a new table for each grain
• Depending on (translation+omega) or (translation+OmegaCalc) -> tth/eta/k/gv/hr/drlv
• A peak to grain assignment matrix should be very sparse. Currently only one grain per peak. It would help for twins and duplicates to store the N grains per peak which might be able to index.

The rest of the this would imply a bit of reorganisation to update the geometry to pull out detector versus diffractometer + grain computations.

Note the cImageD11 function score_gvec_z (needs testing for omega error effect)

from imaged11.

Some thoughts on this (might be nonsense):

Let's assume we have a UBI, and some peaks that the indexer knows about.

Let's also assume we have the power to determine the gve and omega angle for each hkl given a list of hkls
Therefore for each UBI, we have a list of theoretical peaks each with [hkl, omega, gve, xl, yl, zl]
We should therefore be able to determine where (in detector space) each peak hits the detector, so we have [fc, sc] for each theoretical peak

We then have three types of coverage to consider:

1. The coverage in two-theta. We probably filtered the peaks given to the indexer. Maybe just take min to max in tth column?
2. The coverage in omega. Did we rotate the sample to that angle during the scan?
3. The coverage on the detector. Did we hit a masked region? We should be able to spatially correct the mask to work out the mask co-ordinates in the lab frame. We could round to the nearest integer pixel and see if the forward-projected pixel for the theoretical peak hits (or is close to) a mask position

I feel that it should then be possible to, given a set of rings, compute the number of expected peaks on the detector with those rings, taking into account the multiplicity, two-theta coverage, omega coverage, and mask coverage.

That gives us the completeness for a given range of hkls comparing the observed vs expected peaks for a given UBI.

Are there any important other terms I haven't considered? There may be edge cases like peak overlap, but I feel that a semi-working solution is better than no solution

from imaged11.

@jadball The forward projector is here : https://github.com/FABLE-3DXRD/ImageD11/blob/master/sandbox/forwards_project.py

I think it just misses a refresh and testcase and to add the mask function from #241? And to return a columfile rather than ascii !!!

from imaged11.