Kotlin Notebook crash when creating DataFrame from List<Map> about dataframe HOT 5 OPEN

Working solution

val df = (1..1000).toDataFrame {
    "id" from { it }
    "value" from {"value$it" }
}

from dataframe.

Need to check - does it have the same behaviour in the Gradle projects

from dataframe.

It works fine in gradle projects, so we'll need to check what weird type inference is going on in the Jupyter integration...

It creates a DataFrame like:

However, was this the solution you were looking for?

I suspect you want to create a dataframe with a column id and a column value right? Then indeed @zaleslaw 's solution works great.

Constructing a DataFrame is usually done by column and not by row, as that's how they're stored in memory. That's why all DataFrame creation methods are built the way they are. If you have a List<Map<String, T>> and you want each map to be like a row, you could make something like this:

val df = (1..1000).map {
    mapOf("id" to it, "value" to "value$it")
}.toDataFrame {
    source.map { it["id"] }.toColumn() into "id"
    source.map { it["value"] }.toColumn() into "value"
}

If you really have to construct a DataFrame row by row, in theory you could but it would essentially entail making many small DFs and concatenating them, like:

val df = (1..1000).map {
    mapOf("id" to it, "value" to "value$it")
}.map {
    dataFrameOf(header = it.keys, values = it.values)
}.concat()

from dataframe.

Ah yeah. Good explanation. I just copied some code from ChatGPT as part of testing something else when I saw the crash. So I didn't realize that it did things slightly wrong.

from dataframe.

@cmelchior could we close?

from dataframe.