Discussion: Encoding and serializiation about saito-rust HOT 7 CLOSED

I will just leave it here. Benchmarking code of protobuf and bincode. https://gist.github.com/ilblackdragon/88ab377bf0da7ff721af80cb8a11fe28

from saito-rust.

Tagging @batzor here for maybe more feedback on this issue in general

Bincode

I'm a big fan of using default libs to do stuff. But yesterday I tried finding anything JS-library-ish that could handle Rust-bincode. Just to see if there's something in that direction. Didn't find anything that looked hopeful.

One of them in my mind is will the Rust version be the only version that is allowed to run consensus?

I could imagine that locking in now to this might be an issue later. Especially if there's no bincode support in other languages (But maybe there's chance this will be added in the future?).
How to send/receive data with a npm-module later on?

Related bincode-repo issues:

• Serialization in other languages #266
• Tagged enum support / JSON interoperability #272

Protocol buffers

For the lack of alternatives this sounds most reasonable

Related projects:

• https://github.com/stepancheg/rust-protobuf
• https://github.com/protobufjs

Custom

I am not a big fan of implementing such things in a custom way, mostly because of the mentioned cons

from saito-rust.

One big downside of protobufs is that the entire protobuf spec becomes part of our spec. The upside is we might get considerable performance benefits.

I think bincode isn't a viable long-term solution.

"Would require recreating it for each new implementation in another language". I view this as a "pro" for custom serializers, not a con. The encoding scheme must be implementable in arbitrary languages.

from saito-rust.

Just though I'd mention, benchmarking and tests of these different solutions exist in the serialization branch here in this repo for anyone who's curious or wishes to help us extend testing.

from saito-rust.

Linking the mentioned branches:

• https://github.com/SaitoTech/saito-rust/tree/serialization_play
• https://github.com/SaitoTech/saito-rust/tree/serialization_test

from saito-rust.

Can this be closed?

from saito-rust.

Thank you for the discussion everyone!

We are implementing custom serialization at this point.

from saito-rust.