Comments (5)
That is exactly what I was asking for. I honestly did not know that loops are available within a pipeline function.
They are available as you still work on data nodes and it just creates a data processing graph (static). However, a dynamic loop, where for example the number of iterations depends on the other data node is not supported.
from dali.
Hi @Tomsen1410,
Is it possible to build such a dynamic pipeline? Or should I rather stack two (or more) pipes together in this case?
If I understand your ask correctly want to do something like:
my_input_data_tuple = fn.external_source(...,, num_outputs=N)
my_output_data = []
for data in my_input_data_tuple :
my_output_data.append(fn.my_cool_op(data))
return *my_output_data
If not can you please provide a pseudocode of what you want to achieve?
from dali.
Hi @Tomsen1410,
Thank you for reaching out.
If I understand your use case correctly what you can do it that you can check if outs
is just a single data note or an iterable and then use the first element of outs outs[0]
.
from dali.
Thanks @JanuszL!
That is true, I honestly did not think about that...
But for the future, is it somehow possible to iterate over lists or tuples within a pipeline?
Let's say I want to build a pipeline for decoding various media types and I input a list of raw bytes from an external source. The first two list entries might be image bytes, and the last three list entries might be video bytes. Is it possible to build such a dynamic pipeline? Or should I rather stack two (or more) pipes together in this case?
from dali.
That is exactly what I was asking for. I honestly did not know that loops are available within a pipeline function.
from dali.
Related Issues (20)
- A100 hardware decoder HOT 1
- Extract motion vectors HOT 7
- Segmentation fault when using 'mixed' HOT 5
- Bbox Pruning Too Aggressive? HOT 5
- Indexing video with binary mask HOT 1
- source_info tensor not guaranteed to contain correct data HOT 1
- 16 bit gray scale Image read error HOT 1
- COCO Reader pixelwise_masks Emtpy Output HOT 7
- Dali on Jetson: nvidia.dali.fn.readers.video_resize is missing HOT 4
- Numpy reader test (GDS) HOT 4
- How to add a scalar value to the loader? HOT 1
- Can DALI be integrated into HuggingFace Trainer? HOT 9
- Bug in creating `TensorGPU` when `stream` key is `None` in CUDA array interface HOT 2
- Configure max image size HOT 3
- Webdataset reader behavior with many sources HOT 1
- ModuleNotFoundError: No module named 'nvidia.dali.python_function_plugin' HOT 3
- Speed up Dino with DALI HOT 3
- error using webdataset
- webdataset cannot stop cycling at end of epoch HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dali.