Runtime error using Numpy readers: Unknown Numpy type c8 about dali HOT 4 OPEN

Hi @thomas-bouvier,

Thank you for providing more background regarding your use case.
Based on how we understand the ask, this is to make the numpy reader capable of loading files with complex number types, adding a complex type representation to DALI, and reviewing the available operations to see if they should support complex types.
We will evaluate the request and see how it fits our roadmap, in the meantime can you try Python operator on the GPU which can wrap the loading part and the conversion from the complex numbers to the real one, then you can process the data further using the existing DALI operators.

from dali.

Hi @thomas-bouvier,

While technically feasible (still challenging) I'm not sure if we see a good use case for it.
It would be very helpful if you could describe what is the workflow you want to use this feature for.
From the DALI point of view, we would need to create an internal representation (either one tensor that stores this type or two for real and img part) of it and think which operator should support it.

from dali.

Hello @JanuszL, thank you for the feedback (and sorry for the delay).

I understand that this would be an advanced feature, probably not useful to many. Still, let me explain my use case for complex-floating point types.

I am working on xray imaging using diffraction patterns as input data. These diffraction patterns are acquired by a synchrotron light source. A DNN model is used to reconstruct 2 images for every single diffraction pattern : a structure image (amplitude) and a phase image. This is where working with floating point types is needed: the ground-truth data is a collection of numpy arrays containing complex types, from which one can calculate the ground-truth structure and phase above.

As of now, this is how I calculate the ground-truth structure and phase images from the complex numpy arrays rspace_data (raw ground-truth data):

task_ampli_data = []
task_phase_data = []
for i, _ in enumerate(tqdm(file_paths, desc=f"Loading {len(file_paths)} perspectives")):
    # Complex data
    rspace_data = np.load(rspace_paths[i])
    # Calculating the phase and amplitude from the real-space data
    ampli_data = np.abs(rspace_data)
    phase_data = np.angle(rspace_data)

    # Concatenating scan position(s) for this task
    ...
    task_ampli_data.extend(ampli_data[idx][shard_offset : shard_offset + shard_size])
    task_phase_data.extend(phase_data[idx][shard_offset : shard_offset + shard_size])

task_ampli_data = np.array(task_ampli_data, dtype=np.float32)
task_phase_data = np.array(task_phase_data, dtype=np.float32)
taskset = (task_diff_data, task_ampli_data, task_phase_data)

The shape of an individual rspace_data npy file is 1000x1x256x256, giving ampli_data and phase_data of same shapes.

Ideally, I would be able to write the following:

@pipeline_def(batch_size=1, num_threads=1, device_id=device_id)
def input_pipeline():
    file_paths = taskset.get_raw_samples()[0]
    rspace_paths = [f"{p}/patched_psi.npy" for p in file_paths]

    # This npy file contains complex numbers, unfortunately not
    # supported by DALI
    rspace_data = fn.readers.numpy(
        device="gpu",
        files=rspace_paths,
        shard_id=shard_id,
        num_shards=num_shards,
    )

Later on, we could leverage abs and angle operators to calculate ampli_data and phase_data in the pipeline directly.

This is just an idea, your feedback is appreciated :)

from dali.

Thank you for the feedback. Here is an archive containing 5 diffraction patterns patterns.npy.tar.gz.

from dali.