Correctly implement arrow-stream about lakeapi HOT 4 CLOSED

why? :) I used that one for streamlit. is it the same as arrow? or does it not have any true benefits?

from lakeapi.

We might have to check how ROAPI is doing it. The idea would be to stream the arrow batches.

from lakeapi.

Might be more of a decoding issue. To get a dataset you'd need to split the response to multiple files. I can try to do a sample

from lakeapi.

There are two different Mime types

vnd.apache.arrow.stream
vpn.apache.arrow.file

https://arrow.apache.org/faq/

We recommend “.arrow” as the IPC file format file extension:

https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format

and “.arrows” for the IPC streaming format file extension:

https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format

format=arrows
format=arrow

from lakeapi.