GitHub Roadmap 2024 about dali HOT 4 OPEN

I've recently come across some few gaps in datatype compatibility.

For example I have a int32 mask, I want to perform the usual random rotate -> scale -> flip -> ... etc pipeline for augmentation. Scale does not support int32, but does support uint16....however flip does not support uint16. FP16 is also relatively weak in areas.

I think an audit of currently accepted datatypes in some of the operations, and check the reason why it is missing (maybe an NPPI impl isn't available), and if there is no particular reason, how it can be unblocked.

from dali.

@5had3z Thanks for pointing this out. Some of the types were probably overlooked and we can take a look at what we can do. Some others were trimmed to limit the binary size. Each additional type typically translates into a kernel instantiation. If there are multiple types (multiple inputs, outputs) the size quickly gets out of control. Finally, as you pointed out, in some cases we depend on external libraries and are limited by what they support.

from dali.

@5had3z Some others were trimmed to limit the binary size. Each additional type typically translates into a kernel instantiation. If there are multiple types (multiple inputs, outputs) the size quickly gets out of control.

I have zero experience with this, so have no idea on the feasibility. But would there perhaps be a way to JIT operations that have these large parameter spaces? Some ops could still be done ahead of time for faster pipeline start-up times for the common cases, and uninstantiated ones done on the fly.

Another catch-22 I have is that I want my pytorch DALIGenericIterator to know the "size" of my dataset, but if I use an external_source that is a python function, I don't get to use reader_name and I get a bunch of complaints about using size and last_batch_padded (the pipeline itself seemingly works fine anyway). I would be comfortable in writing my custom source dataset in c++, but I haven't looked into it deeply since there isn't pre-written tutorial yet.

from dali.

@5had3z

I have zero experience with this, so have no idea on the feasibility. But would there perhaps be a way to JIT operations that have these large parameter spaces? Some ops could still be done ahead of time for faster pipeline start-up times for the common cases, and uninstantiated ones done on the fly.

I think you hit the spot. The runtime compilation is getting more and more established in the industry and DALI is also looking into different implementations of such capabilities. For now you can use CuPy or Numba to define operation in Python and have it compiled. We are also working on exploring approaches to the native operations DALI currently supports.

Another catch-22 I have is that I want my pytorch DALIGenericIterator to know the "size" of my dataset, but if I use an external_source that is a python function, I don't get to use reader_name and I get a bunch of complaints about using size and last_batch_padded (the pipeline itself seemingly works fine anyway). I would be comfortable in writing my custom source dataset in c++, but I haven't looked into it deeply since there isn't pre-written tutorial yet.

I'm afraid that is one of the limitations of the external source operator, while it gives you the freedom of returning any number of batches it strips the iterator of the knowledge of any samples/batches is may expect. The most flexible approach, in this case, is to always return full batches of data from the external source and just pad it with either a duplicated sample or ones that are randomly selected from the whole data set.

from dali.