Why does having inconsistent input and output dimensions in a Halide generator significantly slow down performance? about halide HOT 4 OPEN

In Example1Generator the innermost dimension in memory of the output buffer is 'c'. That's a small dimension - probably length five in practice given that you're muxing together five things. That means any use of SIMD (vectorization) is going to need to do a strided store to memory instead of a dense vector store, which is very bad for performance.

So if Example1Generator is slow it's probably due to the memory layout of your output buffer. It looks like you're using an autoscheduler, but this is not something an autoscheduler can help with because the memory layout of the output buffer is controlled by the caller.

A fast version of Example1Generator would look like:

    Var x("x"), y("y"), c("c");

    Func f1;
    f1(x, y) = input(x, y) + 10;
    Func f2;
    f2(x, y) = input(x, y) + 20;
    Func f3;
    f3(x, y) = input(x, y) + 30;
    Func f4;
    f4(x, y) = input(x, y) + 40;
    Func f5;
    f5(x, y) = input(x, y) + 50;

    output(x, y, c) = mux(c, {f1(x, y), f2(x, y), f3(x, y), f4(x, y), f5(x, y)});

    output.reorder(c, x, y).bound(c, 0, 5).unroll(c).vectorize(x, 16).parallel(y, 8);

Note that 'reorder' just reorders the loops, not the memory layout.

from halide.

It's nothing to do with dimensionality. It's because calling Example2Generator five times reads the input from memory five times, and calling Example1Generator once reads the input from memory once (and then does five things with it). You'd see the same effect in C.

from halide.

Calling Example2Generator once takes about 3ms, but calling Example1Generator once takes 160ms. This is the confusing part for me.

from halide.

Thank you very much for your suggestion, it's very helpful.

from halide.