[FEA] Support a Scalar only column about cudf HOT 4 OPEN

This sounds like a special case of a run-length encoded column.

from cudf.

Could you elaborate on which operations would use this and how they would work?
So much of our code checks to make sure column-types match.
For example, even dictionary columns generally operate only against other dictionary columns.

It may be easier to overload certain APIs to accept a scalar parameter instead of a column.

from cudf.

This sounds like a special case of a run-length encoded column.

Yes I would accept a run length encoded column too.

Could you elaborate on which operations would use this and how they would work?

That is kind of hard to do right now. Conceptually we pass tables around in Spark between different operators. It is not really a table, as it is Spark specific, but we treat them the same and convert back and forth between them everywhere. I was hoping to be able to keep this abstraction, and just allow us to have a scalar column, or run length encoded column as the case may be.

If there is no simple way to add an abstraction to get this, then we should go back and see how much of this we can do ourselves outside of using a table everywhere.

from cudf.

I assume the scalar column would only be for fixed-width-types and strings. There are certain APIs where the internal implementation converts (non-nested) inputs to iterators and here a scalar could easily be represented as a column using a constant iterator. And so these APIs could have an alternate signature to plumb the scalar (column or rle column) through.
Of course, not all internal implementations are coded this way so the amount of effort here depends on what APIs need to be targeted.

from cudf.