Get audio data from external data sources and start iteration about dali HOT 7 CLOSED

Thank you very much for your response. I have successfully resolved my issue. I will close this matter now and once again thank you for your advice.

from dali.

Hi @Tpcsy,

Thank you for reaching out.
Can you try:

             b=np.frombuffer(f.read(), dtype=np.uint8)
             # print(b)
             batch.append(b)
         return [batch] 

As the documentation states:

Depending on the value of num_outputs, the source can supply one or more data items. The data item can be a whole batch (default) or a single batch entry (when batch==False). If num_outputs is not set, the source is expected to return one item (a batch or a sample). If this value is specified (even if its value is 1), the data is expected to a be tuple, or list, where each element corresponds to respective return value of the external_source.

from dali.

from dali.

Hi @JanuszL
Thank you very much for your reply.
I tried the changes you suggested, and it resulted in the following error:
RuntimeError: [/opt/dali/dali/pipeline/data/tensor_list.cc:1012] Assert on "IsDenseTensor()" failed: The batch must be representable as a tensor - it must have uniform shape and be allocated in contiguous memory.
I'm not sure if this is related to the different sizes of my audio files.

from dali.

Hi @Tpcsy,

I'm not sure if this is related to the different sizes of my audio files.

DALI expects (similarly to Torch data processing) samples to have uniform shapes in the batch. What you can do is to either trim/pad them or use 'DALIRaggedIterator' that will allow you to return nonuniform batch as a set of Torch tensors.

from dali.

Thank you very much for your suggestion. I tried the DALIRaggedIterator as you mentioned and successfully retrieved a batch of data. Now, I want to perform cropping operations on my tensors in the pipeline. Due to the nature of my task, each audio clip needs a different cropping position, and I need to return string-type data from an external source to guide the cropping operation on my audio tensors. However, I couldn't find a method in the pipeline that can accept and use string data. Can you please advise me on how to proceed? Thanks again for your help.

from dali.

I need to return string-type data from an external source to guide the cropping operation on my audio tensors

I'm not sure if I understand your idea correctly.
What you should do is to return the start and size of the cropping window and use the slice operator.

from dali.