Deserialization should be considered in benchmark about ch-bench HOT 5 CLOSED

Agree. Internally there are already a de-serialization, but uint64 values are not available to user.

Probably we can count some very fast hash on those values (or just sum them up?) on all values to force de-serialization?

Not sure how to deal with it in general.

from ch-bench.

I think we probably should avoid extra calculation, which might be unfair for programming languages like Java and Python :p i guess it's sufficient enough to call driver-specific method to make sure bytes were converted to uint64, either explicitly or implicitly. For Java, below snippets will do the work.

https://github.com/go-faster/ch-bench/blob/c2627b1d0fa1a8abc7d3560816d3b919b983600b/ch-bench-java/Main.java#L60

https://github.com/go-faster/ch-bench/blob/c2627b1d0fa1a8abc7d3560816d3b919b983600b/ch-bench-java/Main.java#L88-L90

https://github.com/go-faster/ch-bench/blob/c2627b1d0fa1a8abc7d3560816d3b919b983600b/ch-bench-java/Main.java#L104-L106

from ch-bench.

AFAIK ch-bench-official is doing it implicitly here:
https://github.com/go-faster/ch-bench/blob/c2627b1d0fa1a8abc7d3560816d3b919b983600b/ch-bench-official/main.go#L30-L33

Same as uptrace:
https://github.com/go-faster/ch-bench/blob/c2627b1d0fa1a8abc7d3560816d3b919b983600b/ch-bench-uptrace/main.go#L24-L28

Personally I would prefer something like that:

var count int
for rows.Next() {
	var value uint64
	if err := rows.Scan(&value); err != nil {
		return err
	}
	count++
}

But this will dramatically increase time from 1.388s to 22.567s which is unfortunate (but actually more realistic). This would not affect ch-go and chconn, because they don't rely on reflection.

WDYT?

from ch-bench.

Thanks @ernado. It looks like rows.Next() is mainly about reading blocks according to here, while rows.Scan is the deserialization we're talking about - see here.

Yes, I think the change is more accurate although it will increase the time - perhaps the author has better idea.

By the way, I think besides UInt64, we probably should also consider other data types like String, DateTime and Array(Java is extremely slow for processing nested types) etc.. Below is something I tried earlier but never got enough time to fill out all those tiny cells :p

from ch-bench.

we probably should also consider other data types like String, DateTime and Array

never got enough time to fill out all those tiny cells :p

I've tried to do it, but with same result - never got enough time :D

from ch-bench.