Generated code produces a large amount of non_camel_case_types and dead_code warnings, it should be marked allow so that rustc doesn't complain.

Hello. I haven't real experience with capnproto or rust, except per two toy micro proyects. But i wish help to port capnproto to tokio and future-rs. I'll remove the dependencies on gj and gj-io, add tokio and futures-rs and fix compiler errors until the compiler dont scream. Is that good idea?

P.d. sorry by my english

Currently, when you read a message and the traversal limit is exceeded, you get an error like "Message contained out-of-bounds list pointer", which is inaccurate. It should instead say something like "Exceeded message traversal limit".

A 0 is hardcoded in the compiler but used with floating point types as in https://github.com/millere/osiris/blob/3cc9adc1d5fb698b8e1aa6b8dbba07b471395303/net/src/proto.capn and https://github.com/millere/osiris/blob/3cc9adc1d5fb698b8e1aa6b8dbba07b471395303/net/src/proto_capnp.rs#L80

I think this could be great for python (and other) FFI so I have been trying to figure out how to load &[u8] into an actual object. This is what I have so far but I think it's probably misguided.

#[no_mangle]
pub extern fn test_capnproto(thing: *const u8, length: size_t) {
    let array: &[u8] = unsafe{ slice::from_raw_parts(thing, length as usize) };
    let mut message = MallocMessageBuilder::new_default();
    let mut test_date = message.init_root::<date::Builder>();
    { // scope for builder 
        let b = test_date.borrow();
        for (i, elem) in array.iter().enumerate() {
            b.set_data_field(i, *elem);
        }
    }

    println!("{:?}", test_date);
}

Any tips? Perhaps something to do with read_message? I couldn't figure it out from the examples

Hello,

I'd like to:

• use capnproto-rust to build messages in a specific location of mmap'd memory.
• create a capnproto message from a specific location of mmap'd memory.
• limit the size of the message. If while building a message setting some value would go beyond the mmap length, then return some Error. I can then chunk messages myself.
• use SegmentArrayMessageReader

Any tips / pseudo-code on how to do this would be much appreciated.
Thanks!

Edit: I note the C++ code has capnp::FlatArrayMessageReader. Something like this seems to be workable for reading a message from a mmap'd memory segment.

I came across this protocol and it looks absolutely perfect for microcontrollers! I understand some features support dynamically sized types, so cannot be supported on the stack, but was wondering if a subset of this library could support no_std?

If it is possible, but would have to be a separate crate then that would also be useful to know.

Thanks!

The crates documentation discusses build scripts where you would have a git submodule in tree which is built. Usually this is done in a separate -sys crate but there's no requirement. Building the underlying library is done because there is a desire to reduce the dependencies on system wide installed libraries since it results in inconsistent behaviour. It also means that when cargo has built a package, it's ready to go and doesn't result in errors like these.

[Let me know if there's a better place for these questions.]

What's the best way to handle different capnp message structs coming in from the same zmq socket? There doesn't seem to be a way to differentiate them before decoding. I see there's a type_id on the reader and there's also capnp generics, maybe a third approach?

In my previous implementation with json I had a top-level "request" object which contained a type id and a string blob. Then I would use a match on the type id to decode the inner string blob into a struct based on the type of request. (This was a hack to work around rustc-serialize limitations at the time.)

EDIT: Maybe unions? http://stackoverflow.com/questions/28149139/serializing-mutable-state-and-sending-it-asynchronously-over-the-network-with-ne#comment44738481_28156323

I have a schema containing a struct field named data that's a union. That causes a build failure:

error[E0255]: a module named `data` has already been imported in this module
    --> /home/roc/pernosco/code/target/debug/build/rr-trace-3d4d5d2e72ce7323/out/rr_trace_capnp.rs:3240:7
     |
3064 |         use capnp::{text, data, Result};
     |                           ---- previous import of `data` here
...
3240 | /       pub mod data {
3241 | |         #![allow(unused_imports)]
3242 | |         use capnp::capability::{FromClientHook, FromTypelessPipeline};
3243 | |         use capnp::{text, data, Result};
...    |
3458 | |         pub type WhichBuilder<'a,> = Which<Result<primitive_list::Builder<'a,i32>>,Result<struct_list::Builder<'a,::rr_trace_capnp::opened_fd::Owned>>>;
3459 | |       }
     | |_______^ `data` already imported

Hi! In http://youtu.be/A65w-qoyTYg?t=14m57s you say that reading from a Cap'n Proto message is comparable to a struct field access, or at least I've got that impression from your talk.
Now, testing it with a simple benchmark

#[bench] fn capnp_unpack (bencher: &mut Bencher) {
  bencher.bytes = 3;

  let encoded: Vec<u8> = {
    let mut builder = MallocMessageBuilder::new_default();
    { let mut cache_value = builder.init_root::<cache_value::Builder>();
      cache_value.set_value (b"foo"); }
    let mut buf = Vec::with_capacity (128);
    capnp::serialize::write_message (&mut buf, &builder) .unwrap();
    buf};

  bencher.iter (|&:| {
    let mut bytes = encoded.as_slice();
    let reader = capnp::serialize::new_reader (&mut bytes, capnp::ReaderOptions::new()) .unwrap();
    let value = reader.get_root::<cache_value::Reader>();
    assert! (value.get_value() == b"foo");
  });}

which uses the following schema

struct CacheValue {
  expires @0 :UInt64;
  value @1 :Data;
  tags @2 :List(Text);
}

shows that reading from a Cap'n Proto message is much more expensive than a struct access.
I get 3931 ns/iter from it.
(In fact, the LMDB database where I keep this value is faster at returning it from the database (616 ns/iter!) than Cap'n Proto is at decoding it).

Theoretically I'd imagine that the decoding should involve just some memory accesses, but examining the new_reader code I see a couple of Vec allocations there.

So,
Am I doing something wrong?
Is it a state of the art Cap'n Proto performance or will it be improved?

I admit defeat :) Continuing this question from IRC...I've been unable piece together how to appropriately use Owned. I'm not familiar enough with capnp or Rust lifetimes to work out how the various pieces are used, and there aren't many examples in the three code repositories that use Owned.

My question on IRC:

any capnp tips to dealing with ownership? E.g. i want to serialize a timestamp, but to "rebuild" the timestamp on the other end, Chrono needs to take ownership of the u64 from capnp. I keep running into this with anything more advanced than simple primitives

And @dwrensha's response:

capnp::traits::Owned might help: https://github.com/dwrensha/capnproto-rust/blob/f24ab9e95f1467ff023dd3b8a51495aa1637fa4d/src/traits.rs#L41-L56

I think I eventually want to switch message::Reader and message::Builder to be parameterized by a T where T : for<'a> Owned<'a>.

imagine that message::Reader is paramterized by foo::Owned. Then we would have got_root<'a>(&'a self) -> foo::Reader<'a>. The point is that the container doesn't need to worry about lifetimes; we can wait until a self-borrowing accessor method to commit to a lifetime.

er, it would be pub struct Reader<S, T> where S: ReaderSegments, T: for<'a> Owned<'a> {...}
and get_root<'a>(&'a self) -> <T as capnp::traits::Owned<'a>>::Reader

Do you happen to have a small example handy? This seems like a critical part of using Capnp correctly. I'd be happy to contribute back documentation once I figure out how the various pieces work together :)

You can follow a null capnp pointer. But if you try to access some fields in it (in my case a text field) it will trigger a segmentation fault. I'm not sure if the bug is accessing the field, or being able to follow the pointer in the first place.

Example of the bug: https://github.com/csherratt/capnp_crash

Backtrace:

#0  0x000055555557cf84 in capnp::private::endian::u32.Endian::get (self=0x0) at /home/colin/Source/capnproto-rust/src/private/endian.rs:52
#1  0x000055555557d679 in capnp::private::endian::WireValue<T>::get (self=0x0) at /home/colin/Source/capnproto-rust/src/private/endian.rs:29
#2  0x000055555557db39 in capnp::private::layout::WirePointer::is_null (self=0x0) at /home/colin/Source/capnproto-rust/src/private/layout.rs:322
#3  0x0000555555581048 in capnp::private::layout::wire_helpers::read_text_pointer (segment=0x0, reff=0x0, default_value=0x0, default_size=0)
    at /home/colin/Source/capnproto-rust/src/private/layout.rs:1773
#4  0x0000555555581eb2 in capnp::private::layout::PointerReader<'a>::get_text (self=0x7fffffffd5d8, default_value=0x0, default_size=0)
    at /home/colin/Source/capnproto-rust/src/private/layout.rs:1915
#5  0x000055555555c56d in capnp_crash::test_capnp::foo::Reader<'a>::get_txt (self=...) at target/debug/build/capnp_crash-d343ef91834373e5/out/test_capnp.rs:59
#6  0x000055555555cc0a in capnp_crash::main () at src/main.rs:26
#7  0x000055555558c7f5 in rt::unwind::try::try_fn::h9959899096596013704 ()
#8  0x000055555558a479 in __rust_try ()
#9  0x000055555558c4e0 in rt::lang_start::h46248b0221ecaad5blx ()
#10 0x000055555556ba67 in main ()

Hello,

While playing with the lib, I opened a broken file and it crashed with an assert:

---- tests::broken_file stdout ----
    thread 'tests::broken_file' panicked at 'assertion failed: size > 0', /Users/octplane/.multirust/toolchains/nightly/cargo/registry/src/github.com-88ac128001ac3a9a/capnp-0.7.0/src/serialize_packed.rs:101
stack backtrace:
   1:        0x104de2598 - std::sys::backtrace::tracing::imp::write::h9fb600083204ae7f
   2:        0x104de4835 - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::hca543c34f11229ac
   3:        0x104de43bf - std::panicking::default_hook::hc2c969e7453d080c
   4:        0x104dd9466 - std::sys_common::unwind::begin_unwind_inner::h30e12d15ce2b2e25
   5:        0x104d7b6d7 - std::sys_common::unwind::begin_unwind::h65390c63b501d6ed
   6:        0x104d80bb9 - _<capnp..serialize_packed..PackedRead<R> as std..io..Read>::read::h225fe399cdb1fa2c

You can checkout https://github.com/octplane/kiosque at tag ISSUE_CAPNPROTORUST_BROKEN_FILE and run cargo test broken_file.

Schema is at https://github.com/octplane/kiosque/blob/ISSUE_CAPNPROTORUST_BROKEN_FILE/logformat/src/schema.capnp and guilty file is https://github.com/octplane/kiosque/blob/ISSUE_CAPNPROTORUST_BROKEN_FILE/broken/sample148.capnp

I suppose this should be recoverable in someway at the consumer level, hence this issue.

Thanks!

I have a without @1 :Text; field in a message which is (intentionally) not initialised. The result of calling get_without() on the reader for this message varies based on compile settings. This is not ideal.

With --release and lto = true, it returns an error:

Error { kind: Failed, description: "Message contains non-list pointer where text was expected." }

On all other configurations, including --release without lto = true, and non-release, it returns the empty string.

This is, um, not expected.

• Is there any intentional variation in capnproto-rust, or is this arguably a compiler bug?
• What's the expected output? Should get_ error for missing fields?
• What does the error mean?

I've made a sample project, which will demonstrate the problem with cargo run --release. The other cases can be tested with cargo run and by #commenting lto = true in Cargo.toml, and cargo run --release. It's a binary (which does nothing on success) as it doesn't seem to fail in cargo test --release; didn't look why.

git clone https://github.com/FauxFaux/capnp-get-null
cd capnp-get-null
RUST_BACKTRACE=1 cargo run --release

    Finished release [optimized + debuginfo] target(s) in 0.0 secs
     Running `target/release/capnp-get-null`
thread 'main' panicked at 'without: Error { kind: Failed, description: "Message contains non-list pointer where text was expected." }', /checkout/src/libcore/result.rs:906:4
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
...
   6: core::result::unwrap_failed
             at /checkout/src/libcore/macros.rs:41
   7: capnp_get_null::main
             at /checkout/src/libcore/result.rs:799
             at src/main.rs:28
             at src/main.rs:35
   8: main
   9: __libc_start_main
  10: _start

The code which errors is at: main.rs:28

The schema is totally trivial:

@0xbc035dbe0d877227;

struct Foo {
    with    @0 :Text;
    without @1 :Text;
}

Tested on Ubuntu 17.10 (Artful) amd64 with rustc 1.21.0 (3b72af97e 2017-10-09) (stable) and rustc 1.23.0-nightly (8b22e70b2 2017-10-31).

Generic types currently return AnyPointers but should return the actual type. By generic I mean:

struct Container(Value) {
    value @0 :Value;
}

It's prohibitively difficult to use capnproto-rust with non-blocking input/output streams (Read/Write instances that can return WouldBlock. I've been looking at the codebase the last few days to see what it would take to add support for resumable serialization/deserialization. I believe by adding a WouldBlock variant to the capnproto Error type which holds intermediate state it would be possible. So for instance:

enum WouldBlockState {
    ReadPacked(..),
    Read(..),
    WritePacked(..),
    Write(..),
}

enum Error {
    Decode { .. },
    WouldBlock(WouldBlockState),
    Io(io::Error),
}

with additional read/write methods to resume reading a message after a WouldBlock error:

pub fn resume_read_message(WouldBlockState, &mut InputStream, ReaderOptions) -> Result<..>;
pub fn resume_read_message_packed(WouldBlockState, &mut BufferedInputStream, ReaderOptions) -> Result<..>;
pub fn resume_write_message(WouldBlockState, &mut OutputStream, ReaderOptions) -> Result<..>;
pub fn resume_write_message_packed(WouldBlockState, &mut BufferedOutputStream, ReaderOptions) -> Result<..>;

I want to get your thoughts on this before diving too far into the implementation.

(oops, accidentally submitted early)

Here's an example:

test.rs:

extern crate capnp;

pub mod submod {
    #[path = "../test_capnp.rs"]
    pub mod test_capnp;
}

fn main() { }

test.capnp:

@0xb05abd72309922f1;

struct Test {
  test @0 :Int64;

  method @1 :Method;
  enum Method {
    unkown @0;
  }
}

This errors with:

test_capnp.rs:36:40: 36:50 error: failed to resolve. Maybe a missing `extern crate test_capnp`?
test_capnp.rs:36     pub fn get_method(&self) -> Option<::test_capnp::test::method::Reader> {
                                                        ^~~~~~~~~~
test_capnp.rs:36:40: 36:74 error: use of undeclared type name `test_capnp::test::method::Reader`
test_capnp.rs:36     pub fn get_method(&self) -> Option<::test_capnp::test::method::Reader> {
                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_capnp.rs:64:40: 64:50 error: failed to resolve. Maybe a missing `extern crate test_capnp`?
test_capnp.rs:64     pub fn get_method(&self) -> Option<::test_capnp::test::method::Reader> {
                                                        ^~~~~~~~~~
test_capnp.rs:64:40: 64:74 error: use of undeclared type name `test_capnp::test::method::Reader`
test_capnp.rs:64     pub fn get_method(&self) -> Option<::test_capnp::test::method::Reader> {
                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_capnp.rs:68:38: 68:48 error: failed to resolve. Maybe a missing `extern crate test_capnp`?
test_capnp.rs:68     pub fn set_method(&self, value : ::test_capnp::test::method::Reader) {
                                                      ^~~~~~~~~~
test_capnp.rs:68:38: 68:72 error: use of undeclared type name `test_capnp::test::method::Reader`
test_capnp.rs:68     pub fn set_method(&self, value : ::test_capnp::test::method::Reader) {
                                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to 6 previous errors

The problem is that test_capnp.rs is using :: to try to get to the top of the file, but instead that is referring to the top of the crate. Unfortunately I don't think there's any shorthand to get to the top of the local file scope, so to do this, we would need to create a series of super::...s to walk up the current module chain to get to the root.

Currently, the code (especially layout.rs) is horribly unsafe. In fact, it appears less safe than even the C++ version, which uses template metaprogramming to catch overflow at compile time.

While the interface claims to be safe, the amount of unsafe code in the implementation is worrying when security is a concern. Most of the unsafety seems to appear as a result of accessing fields essentially by raw pointer arithmetic. One solution might be to generate a #[repr(packed)] struct type that represents the actual data off of the wire. Accessing this struct is then safe, at least once one has a reference (by necessity shared) to it. Rust provides #[inline] to ensure that the accessors inline away and have zero overhead.

At present this project is in at least one thing deviating from the standard naming conventions encouraged by the Rust developers and used in most Rust projects; method names are camelCase rather than snake_case. (There may be more; I haven't delved.) It would be good to get this following Rust conventions rather than C++ conventions, at least before v1.0.

Please, please add a basic example somewhere.

At the moment I have absolutely no idea how to use any of your stuff.

Hi,

I am able to write a message to a vec, but I am unable to read it back.
This is how I write to vec:

    let mut message1 = capnp::message::Builder::new_default();
    let mut message2 = capnp::message::Builder::new_default();
    {
        let mut transfer = message1.init_root::<Transfer::Builder>();
        // Setters
        transfer.set_from("fromAddress");
        transfer.set_to("toAddress");
        let mut op_code = message2.init_root::<OpCode::Builder>();
        op_code.set_transfer(transfer.borrow_as_reader());
    }
    let mut out:Vec<u8> = Vec::new();
    capnp::serialize_packed::write_message(&mut out, &message1);

Now I am sending 'out' over the network to another computer (this is why I convert to Vec), how can I deserialize it back to 'op_code' on the other side ?

Especially with cargo, generating code using tools can be annoying. Add a syntax extension that will generate the code into a module. With include_str etc, no flexibility is lost.

capnproto-rust redefines some traits / structures that seem to exist in the standard library, such as BufferedReader and BufferedWriter and the corresponding wrappers, but with a few changes.

Is there a particular reason to redefine those traits and structures, such as an insufficent API or too much API instability? Or is it there more for legacy reasons?

As I could see there is no way to set elements using a mutable iterator.

capnproto-rust is currently impenetrable besides the examples. If one is not already familiar with using it in C++, it's very hard to "dive in".

As per this issue: NixOS/nixpkgs#8186

It would seem Cargo.lock is needed to ensure NixOS builds are deterministic. Would you consider adding Cargo.lock to git repo?

Currently only struct_list implements iter. For data_list, text_list, list_list, enum_list and primitive_list it seems to be missing.

Why does get_segments_for_output take a function of the segments as an argument instead of just returning the segments?

Alright, I think I finally know enough to properly phrase this question.

Right now it seems that the only way to create a TypedReader is to build a message using the standard pattern, serialize, deserialize (which then owns the segments), and stuff into the TypedReader.

Is there a way to build the message using OwnedSegments to start with? Or get at the OwnedSegments from the constructed message? If we're using HeapAllocator, the builder presumably owns the data and it could be safe to move from builder into TypedReader.

EDIT: I suspect it has something to do with Builder::get_segments_for_output(), but I'm not sure how to get the OutputSegments -> OwnedSegments

When attempting to build from on Mac OS X 10.9.2, rustc 0.10 (x86_64-apple-darwin) against 2021bf0, build fails with:

rustc -O capnp/lib.rs --out-dir capnp
capnp/lib.rs:16:1: 16:19 error: can't find crate for `libc`
capnp/lib.rs:16 extern crate libc;
                ^~~~~~~~~~~~~~~~~~
error: aborting due to previous error
make: *** [capnp/compilation-marker] Error 101

The commit which causes this problem is in fact 2021bf0 as 999c7c2 builds perfectly fine.

However, from rust master, this is not an issue. This issue is mostly for others who hit the same problem, as I suspect it won't be fixed.

So that it can be used in rust-lang/rustc-serialize
rust-lang-deprecated/rustc-serialize#35

As I see capnproto is all about non copy/ in-place casting so If at any point you need to deep copy all effort is lost. https://github.com/nikomatsakis/rayon is a parallel crate with easy parallel processing but to use it with capnproto without support you need to copy everything losing capnproto purpose.

Hi,

This might be more of a rust question, but how do I stop EzRpcServer? I have a test that starts the server in a child task, and I'd like to stop it when the test finishes. The test below passes, but then it hangs presumably because the server is still running.

#[test]
fn my_test() {
    spawn(proc() {
        let rpc_server = EzRpcServer::new("127.0.0.1:12345").unwrap();
    });
    // Do things.
}

Thanks!

In trying to structure an app so that messages are read from a (TCP) stream, and then passed off to another thread for further processing, I'm running into issues because the message reader is not Send. Is this intended, or perhaps fallout from the opt-in built-in changes? For reference these are the errors I'm seeing when trying to move a message into a new thread:

error: the trait `core::marker::Send` is not implemented for the type `*const capnp::arena::SegmentReader`
error: the trait `core::marker::Send` is not implemented for the type `*const u8`
error: the trait `core::marker::Send` is not implemented for the type `*const capnp::layout::WirePointer`

Is there any reason IntoIterator trait is not implemented anywhere in the code?

It would be useful at least for capnp::primitive_list::Reader.

could you please add capnproto-rust to http://hiho.io/rust-ci/, thanks

The Serde library supports many serialization formats but not capnproto.

One useful feature is the transcode functionality where it is possible to convert from one format to the other.

So, with a bit of hackery using git submodules, capnp-rpc can be made to work as a cargo package as it stands now, without duplicating all the code. I've managed it with brandonson/capnproto-rust-rpc. Unfortunately, ugly hackery is all it is, and I suspect it wouldn't take much to make it topple over and stop working.

The capnp crate works great with cargo, so it's a shame that capnp-rpc doesn't. I'd love to see capnp-rpc moved into its own repo, as that would let it exist as its own cargo package properly, instead of the weird things I've done to sort of get that working.

It'd be good to be able to use this with rustpkg. The most notable change is that capnprust.rs should be renamed to lib.rs, but there may be other changes necessary; I haven't delved into this project's compiler stuff and do not know if it needs anything special. Tests also get done with test.rs being the root file in a crate.

The InputStream and OutputStream traits closely mirror the functionality of the standard library's Read and Write traits, but they are less flexible. In addition to having to wrap Read and Write instances, it makes it impossible to use a type that implements Read and Write (for example TcpStream) in separate InputStreamWrapper and OutputStreamWrapper instances.

I looked at replacing the uses of these traits with their standard library equivalent, or just providing default implementations of them for the standard library types, but it requires some changes to the deepest darkest parts of serialize_packed, so I got a bit scared off.

/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38:5: 38:70 warning: the trait `core::marker::Sized` is not impl[788/1285]
r the type `Self` [E0277]
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38     fn get_from_pointer(reader : &PointerReader<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38:5: 38:70 help: run `rustc --explain E0277` to see a detailed explanatio
n
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38:5: 38:70 note: `Self` does not have a constant size known at compile-ti
me
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38     fn get_from_pointer(reader : &PointerReader<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38:5: 38:70 note: this warning results from recent bug fixes and clarifica
tions; it will become a HARD ERROR in the next release. See RFC 1214 for details.
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38     fn get_from_pointer(reader : &PointerReader<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38:5: 38:70 note: required by `core::result::Result`
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:38     fn get_from_pointer(reader : &PointerReader<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70:5: 70:71 warning: the trait `core::marker::Sized` is not implemented fo
r the type `Self` [E0277]
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70     fn get_from_pointer(builder : PointerBuilder<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70:5: 70:71 help: run `rustc --explain E0277` to see a detailed explanatio
n
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70:5: 70:71 note: `Self` does not have a constant size known at compile-ti
me
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70     fn get_from_pointer(builder : PointerBuilder<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70:5: 70:71 note: this warning results from recent bug fixes and clarifica
tions; it will become a HARD ERROR in the next release. See RFC 1214 for details.
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70     fn get_from_pointer(builder : PointerBuilder<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70:5: 70:71 note: required by `core::result::Result`
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:70     fn get_from_pointer(builder : PointerBuilder<'a>) -> Result<Self>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90:5: 90:76 warning: the trait `core::marker::Sized` is not implemented fo
r the type `Self` [E0277]
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90     fn from_u16(value : u16) -> ::std::result::Result<Self, ::NotInSche
ma>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90:5: 90:76 help: run rustc --explain E0277 to see a detailed explanatio
n
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90:5: 90:76 note: Self does not have a constant size known at compile-ti
me
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90 fn from_u16(value : u16) -> ::std::result::Result<Self, ::NotInSche
ma>;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90:5: 90:76 note: this warning results from recent bug fixes and clarifica
tions; it will become a HARD ERROR in the next release. See RFC 1214 for details.
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90     fn from_u16(value : u16) -> ::std::result::Result<Self, ::NotInSche
ma>;
                                                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90:5: 90:76 note: required by core::result::Result
/home/cmr/.multirust/toolchains/nightly/cargo/registry/src/github.com-0a35038f75765ae4/capnp-0.4.3/src/traits.rs:90 fn from_u16(value : u16) -> ::std::result::Result<Self, ::NotInSche
ma>;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(I'll take a poke at a patch)

I'm getting this error:

failed to run custom build command for `capnp-rpc v0.5.0`
Process didn't exit successfully: `/opt/capnp-raft/target/debug/build/capnp-rpc-1baa9a0f1ce31119/build-script-build` (exit code: 101)
--- stderr
thread '<main>' panicked at 'called `Option::unwrap()` on a `None` value', ../src/libcore/option.rs:365
stack backtrace:
   1:     0x7fb138e90620 - sys::backtrace::tracing::imp::write::he9cf93dccfa17cceNlt
   2:     0x7fb138e931e5 - panicking::log_panic::closure.39173
   3:     0x7fb138e92c31 - panicking::log_panic::hd9f4aa0cbdff587aZlx
   4:     0x7fb138e79f93 - sys_common::unwind::begin_unwind_inner::h90fc1a96c34cff35eds
   5:     0x7fb138e7a608 - sys_common::unwind::begin_unwind_fmt::hfca8a38c37457f6fkcs
   6:     0x7fb138e8fcf1 - rust_begin_unwind
   7:     0x7fb138ec06bf - panicking::panic_fmt::h19598b35d555b127wZK
   8:     0x7fb138ebc7d8 - panicking::panic::hfefb33ce32c56da43XK
   9:     0x7fb138de0f35 - option::Option<T>::unwrap::h1607710876796834354
                        at ../src/libcore/macros.rs:20
  10:     0x7fb138dde1c0 - main::h7000649270783f3cfaa
                        at /root/.cargo/registry/src/github.com-0a35038f75765ae4/capnp-rpc-0.5.0/build.rs:11
  11:     0x7fb138e92974 - sys_common::unwind::try::try_fn::h6140512138114007296
  12:     0x7fb138e8fb58 - __rust_try
  13:     0x7fb138e9260f - rt::lang_start::h21dafd243738f0d57ix
  14:     0x7fb138deffe9 - main
  15:     0x7fb137fcdaf4 - __libc_start_main
  16:     0x7fb138ddde78 - <unknown>
  17:                0x0 - <unknown>

I've tested both stable and nightly, on Windows and Linux.

If I have a schema like this

struct Foo {
  bar :union {
    // omitted
  }
}

the autogenerated code for Foo::Builder includes a method called init_bar, which consumes Foo::Builder. I'm new to rust, but this seems unideal to me for a couple reasons:

1. If Foo has other values I either have to set them before calling init_bar or I have to create another Foo::Builder from the containing buffer after setting up bar.
2. If Foo::Builder is borrowed in the current scope I can't call init_bar

Was this intentional or is it a bug?

As I could see to get a memory aligned [u64] in capnproto you must use a primitive list with T = u64/UInt64 but them it is unclear how to get access to a slice of it

The c++ api has orphan support are there plans to adding it to the rust api? It seems like the only way to do a bottom-up building of a capn'proto message.

It'd be great if the documentation for the project was available online. You can easily integrate this into your CI automatically in the .travis.yml file if you are able to generate an application token and secure it.

I wrote about how to do this here, it should only take a few minutes if you already have the travis gem installed.

The RPC repo could also benefit from this.

I'm writing a program which wants to read data from files using mmap and capnp. This is not quite memory safe, for the subtle reason[1] that capnp-rust's pointer validation is potentially not TOCTOU safe; if another process makes a stray write into the file (unlikely, but I'd like to cover all valid error situations), it could cause a reader to validate a valid pointer, then use an invalid one resulting in random memory accesses.

If capnp-rust were modified to be TOCTOU-safe when validating pointers, and all reads from the secoment were made into unordered atomic loads[2] to prevent the optimizer from introducing new TOCTOUs, then capnp-rust would be able to provide memory safety even when reading from raw pointers that might be concurrently modified.

[1] There is also the not-subtle reason that I/O errors result in SIGBUS, but I'm tackling that as a separate problem. There are several possible approaches.

[2] std::intrinsics::atomic_load_unordered() exists; unfortunately there's no stable wrapper for this in libstd (you can transmute to AtomicUsize, but that doesn't work for smaller types and only provides Monotonic, not Unordered).

The Rust team yesterday accepted this RFC on a standard Allocator interface. It hasn't been implemented yet, but I was taking a look and it seems like it would cover the needs of the current Allocator trait in capnproto-rust - we can use the Layout struct to provide the same size + alignment properties that Word currently has.

When this becomes implemented in the stdlib, would you be interested in using it instead of/to back the current Allocator scheme in this crate? I'd be interested in working on a PR for this in the future.