Consider renaming "Arrow" case? about db-benchmark HOT 12 OPEN

People wanted to know where is arrow now. Therefore it has been added as is now

The main issue is that there is no such thing as "arrow" or "arrow engine" when it comes to benchmarking libraries. There is a single Arrow specification, but then there are many implementations of this specification, with varying degree of scope and performance.

The dplyr interface provided by the R arrow package is one such implementation, but eg Polars or Datafusion are just as much "arrow" while using an entirely different implementation that doesn't share any code with the R arrow package.

So I think renaming the current dplyr benchmark from "arrow" to the suggested "dplyr-arrow" (or "arrow-dplyr") makes sense.

from db-benchmark.

Thank you for your quick reply.

I do not really understand the argument: arrow is not a query engine, and it does not implement a "group by" or a "join": a group by is an operation, not a format.

In my opinion it is misleading to write "Arrow" , as it gives the sense that the arrow format is very slow, when arrow is not even something that "runs"; it is like saying that "parquet" or "ORC" are slow.

Making the case use more or less arrow automatically does not make it be more or less arrow; query engines and formats are fundamentally different notions.

Disambiguating which specific query engine implementation is being benchmarked helps users deciding what technology to adopt, and portraying Arrow as a query engine only adds confusion to this decision-making process.

I would re-consider opening this issue, at the very least to offer some time for other members of the community to weight in.

from db-benchmark.

Thank you for suggestion. It is arrow as it is now. It will automatically be using more arrow over time, without any adjustments (assuming that arrow API will no be changing). People were asking for adding arrow, therefore it has been added as is as of now.

from db-benchmark.

I'm a user not involved in neither dplyr nor arrow and the current state of names, for me, is confusing. Maybe we should trust @jorgecarleitao with this, as he is a PMC of the Apache Arrow project.

As a use, I also enjoy those benchmarks very much!

from db-benchmark.

@wjones127 thank you for that info. Agree about renaming. Unfortunately I am not maintainer anymore and you have to contact h2o support about any changes in this project.

from db-benchmark.

Arrow will implement groupby and join in future. Also note there is explanation at the bottom of the report page mentioning that. Arrow is query engine (or will be), it uses feather format.

from db-benchmark.

If it uses in fact dplyr, why not call it "dplyr" then? Maybe "dplyr on arrow table" and the other one "dplyr on tibble"?

from db-benchmark.

Because it is arrow as it is now. And automatically will use more of arrow engine over time, once it will be ready. Fallback to dplyr is built in to arrow package, I did not set any of these. Note that fallback doesn't yet work for join.

from db-benchmark.

Reason for adding arrow was not to show that it is as slow as dplyr but to address requests from community. People wanted to know where is arrow now. Therefore it has been added as is now (but automatically will use arrow engine more as fallback happens inside arrow).

from db-benchmark.

@jangorecki , can you describe what "arrow engine" is? What is its source code, what are their capabilities?

from db-benchmark.

I think it's great that Arrow is included, but please include other implementations. For example, it would be great to compare Datafusion to dplyr on Arrow, and similarly compare Ballista to Spark.

from db-benchmark.

The Arrow C++ engine (that supports R arrow's dplyr functionality) has now been named Acero to differentiate it from other Arrow-based engines. We can rename the benchmark to that.

from db-benchmark.