Request for a simple and complete CNN example about fold HOT 2 CLOSED

I think it's important to clarify here what Fold can and can't do. Fold is intended for models over structured data -- e.g. trees, graphs, sequences, records, etc. TensorFlow itself does not provide native support for data structures of this form, nor does it provide the necessary control-flow (e.g. recursion) to implement such models. Fold fills that gap.

Fold also implements "dynamic batching", which means that if you have tensor operations that can be batched together -- i.e. a bunch calls to the same operation that all process data of the same tensor dimensions -- then it will combine that data into a batch for efficiency. This commonly arises in tree models. If you have a parse tree where the leaves are words, and you want to embed each word, Fold will batch together all of the words and issue a single call to the embedding op.

2D and 1D convolutions don't necessarily fit into this category. A 2D convolution is an operation on tensors, not on trees or graphs, so Fold doesn't provide any capabilities beyond what TensorFlow supports. In particular, if you have a bunch of images of different sizes, and want to batch them together, then Fold can't do it. That would require implementing a new convolution kernel in CUDA.

For 1D convolutions, if the input data is a sequence, then you can use the NGrams block to do a convolution. However, the output of NGrams is another sequence, so you will have to do a Reduce (sum, max, or average-pool) to get a fixed size tensor output at the end.

from fold.

@delesley Ok thanks for the clarification. I'm not familiar with the implementation details of CUDA and this wasn't clear to me from the paper and API. I really appreciate the detailed explanation.

from fold.