dlsnxs2cbf: produces invalid files from i04-1 Eiger 9M about dxtbx HOT 10 OPEN

🤨 15 minutes in and already at the "how did this ever work" and "how even does dials work?" phase of the process

from dxtbx.

NeXus file contains the right values:

Ethics-Gradient work :) [main] $ h5ls -rvd ../../Insulin_6_2.nxs/entry/instrument/detector/module/data_size
Opened "../../Insulin_6_2.nxs" with sec2 driver.
entry/instrument/detector/module/data_size Dataset {2/2}
    Location:  1:22972
    Links:     1
    Storage:   8 logical bytes, 8 allocated bytes, 100.00% utilization
    Type:      native int
    Data:
         3108, 3262

from dxtbx.

X-Binary-Size-Fastest-Dimension: {data.focus()[1]}
X-Binary-Size-Second-Dimension: {data.focus()[0]}

is in the f-string => I would expect this to be correct - data comes from

(data,) = dxtbx.nexus.get_raw_data(nxdata, nxdetector, j)

=> can start from there

from dxtbx.

Ethics-Gradient i04-1-run3-ins :) [main] $ dials.import Insulin_6_2.nxs ; dials.show imported.expt | grep image_size
DIALS (2018) Acta Cryst. D74, 85-97. https://doi.org/10.1107/S2059798317017235
DIALS 3.dev.1103-gea4fc09a8
The following parameters have been modified:

input {
  experiments = <image files>
}

--------------------------------------------------------------------------------
  format: <class 'dxtbx.format.FormatNXmxDLS.FormatNXmxDLS'>
  template: /Users/graeme/data/i04-1-run3-ins/Insulin_6_2.nxs:1:1800
  num images: 1800
  sequences:
    still:    0
    sweep:    1
  num stills: 0
--------------------------------------------------------------------------------
Writing experiments to imported.expt
  image_size: {3108,3262}

from dxtbx.

This does not go through the usual dxtbx format mechanism

from dxtbx.

        # dxtbx requires image size in the order fast, slow - which is the reverse of what
        # is stored in module.data_size. Additionally, data_size can have more than 2
        # dimensions, for multi-module detectors. So take the last two dimensions and reverse
        # them.  Examples:
        # [1,2,3]   --> (3, 2)
        # [1,2]     --> (2, 1)
        image_size = (int(module.data_size[-1]), int(module.data_size[-2]))

deep sustained sighing sounds

from dxtbx.

To confirm: the dxtbx view of image size is indeed, fast, slow

Ethics-Gradient work :( [main] $ python3
Python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:25:13) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from dxtbx.model.experiment_list import ExperimentList
>>> el = ExperimentList.from_file("imported.expt")
>>> d = el[0].detector
>>> m = d[0]
>>> m.get_image_size()
(3108, 3262)

from dxtbx.

OK, there are some definite sources of doubt and uncertainty surrounding the module shape - switch or not to switch? If commented out, we create CBF files which contain

X-Binary-Size-Fastest-Dimension: 3108
X-Binary-Size-Second-Dimension: 3262

Dectris say (W x H) 3,108 x 3,262 which is consistent

from dxtbx.

With changes herein XDS is a very happy camper => need to understand why we are doing this.

@phyy-nx we (@benjaminhwilliams and I) note you have recently been exploring some of this terrain and may hold opinions. What we have at the moment is comprehensively wrong

from dxtbx.

This nerdsniped me big time at the end of the day :) Working through my thought process:

• If someone were to take a photo of the detector, would it be taller than it is wide? In which case WxH is 3,108 x 3,262? Ima guess yes, based on how I've seen these.
• The NeXus field /entry/instrument/detector/module/data_size is 3108, 3262, that's defined as (slow, fast), or (W,H) if the above is true.
• Just to be sure, what is the shape of the data in /entry/data/data?
• So, now we move to dials.show, which is returning {3108,3262}. That should be (fast, slow), so that's backwards!
• The comment you cited is definitely right, sighing noises aside :)
• Is this line of code getting hit? Maybe it shouldn't be?
• If /entry/instrument/detector/module/data_size is right, then that cbf header would be backwards for sure

I'll stop here and await answers. Note, I have an active DLS account if you want to DM me a file path!

from dxtbx.