Consider merging with Serde about capnproto-rust HOT 11 CLOSED

Any news on this?

from capnproto-rust.

Hi!

My impression is that Cap'n Proto would not be a very good fit for Serde integration, mainly because Cap'n Proto wants to define data types in schemas external to Rust code. I have an old post on this topic. (The libserialize discussed there is now called rustc_serialize, if I remember correctly.)

Do you have a specific use case in mind?

from capnproto-rust.

I'd think cap'n proto could generate the serde attributes from its schema, maybe useful if someone must work around company policies like always use json or xml for such and such.

from capnproto-rust.

Good point about the schema, I initially thought that capnpc-rust could generate the needed structs with Serde's annotations & any runtime code. Not sure if this is simple, especially considering that Serde does not support protocol buffers.

My use case is converting from JSON to Capnproto as an internal representation for microservices. Using the Serde lib could potentially make the transcoding with less code.

If you think that this is something that does not make sense for this project, feel free to close this issue.

from capnproto-rust.

I do think think that it would be good to have JSON <--> Cap'n Proto support in capnproto-rust, but I'm not sure that it would make sense to involve Serde. I would probably start by looking at how capnproto-c++ supports JSON encoding and decoding.

from capnproto-rust.

Thanks for feedback, closing this issue...

from capnproto-rust.

Today I came across serde-protobuf. I'm not sure exactly what it does, but it seems relevant to this discussion.

from capnproto-rust.

Do you have a plan to support JSON encoding and decoding in capnproto-rust?

from capnproto-rust.

@piaoger It would be good to have, but I don't have any immediate plans to work on it.

I think the way forward would be to implement dynamic reflection as in the C++ reference implementation, so that to_json() and from_json() functions could work for any type of message without needing any extra generated code.

from capnproto-rust.

Cap'n Proto works with aligned data, so it has the capability to return things like &[u32] on little-endian systems, which can be crucial for performance.

Serde on the other hand is capable of zero-copy deserializing &[u8] (and &str) but not anything wider than that. You would therefore be losing out on certain performance characteristics and zero-copy deserialization by trying to fit Cap'n Proto into Serde's data model.

It is possible to model slices of complex types on top of byte buffers by using unaligned reads, which incurs a small performance penalty, but in practice this penalty is often in the noise. The zerovec crate provides these abstractions so that you can use zero-copy deserialization of complex types with Serde. (Disclaimer: I am a maintainer of the zerovec crate.)

from capnproto-rust.

As an aside, we discussed multi- architecture layout ideas in https://internals.rust-lang.org/t/multi-architecture-layout/18538 which likely never happens in rustc, but maybe gives someone ideas.

from capnproto-rust.