Is there an option/widget that allows for the expansion/collapsing of sub-trees or the whole tree?

Can we add a list of the current environment in which you are working? Package dependencies etc...

There are actually three options that we want:

• offset
• max frames
• max total frames?

This error only occurs if you start the hexrd gui in the hexrdgui/hexrd/ui directory. You get the following error:

Traceback (most recent call last):
  File "/home/patrick/virtualenvs/hexrd/bin/hexrd", line 11, in <module>
    load_entry_point('hexrd-gui', 'console_scripts', 'hexrd')()
  File "/home/patrick/virtualenvs/hexrd/lib/python3.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/patrick/virtualenvs/hexrd/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "/home/patrick/virtualenvs/hexrd/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/home/patrick/virtualenvs/hexrd/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'hexrd.ui.hexrd'

HDF5 and npz

I guess the units are mm/pixel_size.

Indeed, if you change the pixel_size, it changes the x and y numbers.

@donald-e-boyce Will provide the options we want to support.

Sometimes, only polar calibrations will be able to be performed. Add in a validation check to make sure cartesian calibrations can be done before doing it.

It is considered feasible to do a cartesian calibration if the dot product of detector normals is less than 120 degrees or so.

In working with the calibration module, I realized that the matrix invariants (axis scaled by rotation angle) used for the tilt degrees of freedom in the latest version of hexrd is not a great set of DOF for the user to interact with. The exponential map is computationally expedient and non-singular, but it will be much, much better to expose a set of Euler angles to the user through the GUI interactors for detector tilt. Furthermore, depending on the configuration, different sets of Euler angles will be appropriate. For the current problem, the best choice are extrinsic lab XYZ Euler angles (note that this was the parameter set used in the branch you guys started working with, but we are moving away from that), but in general, on configuration import we would like a selector for the user to specify the set they'd like to use (consult hexrd.xrd.rotations.make_rmat_euler()). While there are 24 possible combinations, we only currently use 2 variants:

• extrinsic XYZ (axes stay fixed; what is currently in the version of hexrd you are using), and
• instrinsic ZXZ (axes transform).

Could there be a selector on the instrument calibration panel for specifying which convention we want to use? Then the selectors will display Euler angles in degrees.

Thoughts?

Allow live updates as a user changes the calibration parameters, currently you have to re-run the calibration.

This is to return to an image view after looking at a calibration.

We should add a "None" option to the "Function" combo box list, as sometimes there will be no distortion functions for the detector.

Also, different functions use different numbers of parameters. The GE_41RT uses 6 parameters, and a new one that will be added next week uses 8 parameters.

Perhaps we should remove the parameter widgets and put a "Configure" button, which pops up a dialog, and then the dialog will provide the edit-able widgets (and the correct number of them).

We could also just change the number of widgets in the instrument calibration panel depending on the currently selected distortion function.

Changes requested by @joelvbernier

Just noticed that the form view is not updating when loading an instrument config file.

@joelvbernier Will provide script that demonstrates the code we can call to this.

Found this bug when toggling back and forth between polar and cartesian views

Traceback (most recent call last):
  File "/Users/bernier2/Documents/GitHub/hexrdgui/hexrd/ui/async_worker.py", line 71, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/bernier2/Documents/GitHub/hexrdgui/hexrd/ui/calibration/polar_plot.py", line 25, in polar_viewer
    return InstrumentViewer(iconfig, images_dict, plane_data)
  File "/Users/bernier2/Documents/GitHub/hexrdgui/hexrd/ui/calibration/polar_plot.py", line 60, in __init__
    self.generate_image()
  File "/Users/bernier2/Documents/GitHub/hexrdgui/hexrd/ui/calibration/polar_plot.py", line 96, in generate_image
    self.add_rings()
  File "/Users/bernier2/Documents/GitHub/hexrdgui/hexrd/ui/calibration/polar_plot.py", line 118, in add_rings
    tth_list, delta_tth=delta_tth, delta_eta=1)
  File "/Users/bernier2/Documents/GitHub/hexrd3/hexrd/instrument.py", line 1944, in make_powder_rings
    eta_centers = eta_edges[:-1] + del_eta
UnboundLocalError: local variable 'del_eta' referenced before assignment

Perhaps this should be designed to follow the layout from the example they gave us.

In that example, the images are in a single dataset in imageseries/images, and the first index of the dataset is the image number.

h5py is already a requirement for hexrd, so hopefully, it won't be too much to make it a requirement for hexrdgui as well, so we can read the HDF5 files.

Calibration on single images

• Lines (rings) from single crystal monochromatic
• Laue pattern from polychromatic beam

Calibration on a series of images

• Rotation series of a single crystal
• Rotation series of polycrystal

Minimum viable product for the first beam allocation:

• Space to specify dark field from an image file
• If you have an image series with more than one image, autogenerate from series
• Ability to load and export image series in npz and hdf5
• Use calibration classes from hexrd in the GUI
• Do single powder and calibration based on image series
• YAML spec provided to load image series from raw files
• Interactor to provide aggregation for median or max through the image stack (from stats, in wx GUI under frame aggregation - single frames, max or median)
• Display total number of saturated pixels in a dialog

May be with a slider.

They mentioned today that we will eventually need to use the ImageSeries classes for handling the image data. In our current hexrd branch, it looks like there are OmegaImageSeries and ProcessedImageSeries classes.

From Joel "I am imagining a "calibration manager" in which we can build the calibration workflow from individual components; for starters, we'll just have calibration by powder patterns (rings), but it would be nice to build a calibration system from 1 or more materials. The process takes an instrument class, and a list of materials classes. There are a few other parameters in the problem (tth bin size, eta bin size -- everything else should be wrapped up in the classes)."

We need to wait for more info (and possibly an example) from Joel, but we soon need to add a panel with options for doing calibration optimization.

During my terminal session today, I ran hexrd and the GUI came up (complained a little) and worked. Shortly after, I ran a git pull on the hexrd repository; it brought things up to date. Then when I tried to run hexrd again, I just got an error message and nothing came up. I am attaching output of those three commands by command number on terminal: 1009 - run hexrd; 1016 - git pull; 1029 - run hexrd again.

1009-gui-error.txt
1016-git-pull.txt
1029-gui-error.txt

At some point, these options should be moved out of the instrument calibration and put somewhere else. They are only really used for single crystals, so they should be an option specifically for single crystal calibration. Perhaps it can be a menu item (such as Edit->Calibration Crystal) or a pop-up for doing single crystal calibration.

Changes requested by @joelvbernier

I just noticed that I cannot enter an abritrary value in the min/max spin controllers (like 100 - 10000). Are these just limits set on the widgets? In general we look at int type images, so a full int32 range catches all possibilities. What if we have 4-byte float images?

For some reason, when in a subplot view of images, if any updates are performed, the subplots move slightly closer together. It looks like they are approaching some kind of minimum combined size.

It's not really a big issue, but it should perhaps be fixed at some point.

We need to receive a single crystal example first, but we will need to make the calibrations work for single crystal data as well.

For single crystal data, the overlays will be dots instead of rings.

Can we promote the panel for the ring overlays to a separate manager ("overlay manager") that will provide the ability to add overlays from multiple materials? Looking forward, there will be 2 additional overlay types I can envision using:

• single crystal Laue patterns; and
• single crystal rotation series.

in addition to the polycrysal ("powder") rings. The single crystal overlay types will require a material plus a calibration crystal spec. They will be point markers instead of lines. Laue patterns are 2-d for single images like the powder patterns with a set of (x, y) detector coordinates specified for each image, while the rotation series overlays are 3-d with (x, y, omega) coordinates, but could be displayed on a omega-projected imageseries (e.g. max over all frames)

It would be ideal to be able to build an overlay set from combinations of these three components, which would likely require hooks to line/marker spec for the user to differentiate them.

Can we rename this "rendering resolution"?

"Calibration" refers only to the optimization problem.

Also, we need the following

• Cartesian tab
  • add box for virtual plane distance
  • optionally add interactor to define the plane normal
• Polar tab
  • add interactors for min and max 2theta values for the rendering (they are hard-coded now I believe). azimuthal range will default to full 360 degrees for now...

See below:

/Users/joel/miniconda2/envs/hexrdgui/lib/python3.7/site-packages/skimage/util/dtype.py:135: UserWarning: Possible precision loss when converting from float64 to uint16
.format(dtypeobj_in, dtypeobj_out))
Segmentation fault: 11

Is there another log that would be more helpful to grab? This has happened a few times when trying to update the cartesian view after changing a controller val.

In the image processing panel, there should be options for modifying the images by doing things such as dark subtraction and flipping.

Currently, both the image and the rings have to be re-generated every time by pressing the "Run Calibration" button. It would be nice if we kept the background image cached and updated the rings live as the user modified their settings.

In the materials panel, rows can be selected or de-selected by clicking on them individually. We should add a way to select multiple rows at once by doing things such as shift-clicking.

For single images, we currently just store the data, but for multiple images, we store them in an imageseries.

It will be nice if we always store images in an imageseries, even when there is just one image, so we can use features such as image processing on single images.

This will also help with #53 for single images.

In the materials dialog, Joel requested more sig figs to be added for a, b, and c. Maybe we should do 5 digits after the decimal?

After toggling the rings display off, it would not return. Mac OS 10.14.5, python 3.7.3

we would like to be able to have the user select the active rings they want from the displayed list by two methods:

1. Using the mouse (ctrl and shft click)
2. By specifying a maximum bragg angle
3. By specifying a minimum d-spacing

The PlaneData class has an "exclusions" attr that is a boolean mask to select a subset of the complete list of generated reflections (as specified currently by the max sum of h^2 + k^2 + l^2). There is also a tThMax attr that is initialized as None, but if set to a float, it treats it as a cutoff on 2theta. I can code up a dspMin attr to handle the 3rd case. Setting PlaneData.exclusions = None resets to display all.

This functionality was in the old wx GUI under the "edit HKLs" button.

'Run Calibration' => 'Cartesian View'
'Run Polar Calibration' => 'Polar View'

It seems like the spin controllers are stuck modifying whatever panel was first modified, not the detector that is currently selected in the detector parameters (i.e. the "active" detector).

Joel mentioned that we need to add options/checkboxes for whether instrument calibration parameters are refinable or not. We discussed that it might be best to add a column to the tree view for this, but not to add checkboxes to the form view.

He provided this image from Maud as an exmaple:

After #79, the cartesian and polar calibrations are no longer working, probably because some instrument configuration settings have changed, and they need to be updated.

For the cartesian calibration, this is the error we currently see:

Traceback (most recent call last):
  File "/home/patrick/virtualenvs/hexrd/src/hexrdgui/hexrd/ui/async_worker.py", line 71, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/patrick/virtualenvs/hexrd/src/hexrdgui/hexrd/ui/calibration/cartesian_plot.py", line 21, in cartesian_viewer
    instr = instrument.HEDMInstrument(instrument_config=iconfig)
  File "/home/patrick/virtualenvs/hexrd/src/hexrd3/hexrd/instrument.py", line 257, in __init__
    tvec=xform['translation'],
KeyError: 'translation'

The polar calibration produces a similar error.

When you are asked to select a path in the HDF5 and you hit cancel, it still moves on to try and load the file anyway.

Add options somewhere to modify the tilt angle conventions that the instrument calibration widgets are using. Perhaps under the Edit menu, or in the instrument calibration widget itself.

This class is used to cast the detector rotation matrix to Euler angles. The default convention that is currently in use is axes_order=‘xyz’, extrinsic=True. Another convention that we should allow for is axes_order='zxz', extrinsic=False.

Changes requested by @joelvbernier

Cartesian resolution should have:

• Pixel size
• Display plane distance

Polar resolution should have:

• Pixel size
• Azimuth Pixel size

Even though a degree symbol is shown in the "Form View", it appears that the detector tilt angles are in radians. We should perhaps ensure that a conversion happens, or just remove the degree symbol and use radians.

The show ranges spinbox currently appears to work properly for updating the GUI.

However, there is a possibility that we may need to hook it up to the active material's planeData.tThWidth value, so that when we modify the spin box, it modifies the planeData.tThWidth value of the active material. And, if you change the active material, it will update the value in the spin box.

I am not sure if this is needed, though.

I've seen the following warning pop up, and it would probably be good to address it sooner than later:

/Users/bernier2/miniconda3/lib/python3.7/site-packages/pywt/_utils.py:6: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable