Comments (13)
Could you say what it is you're trying to do?
If you're trying to read a serialized message, you probably want capnp::serialize::read_message
or capnp::serialize_packed::read_message
I must admit I'm having a little trouble figuring out how to return the message once it's parsed, due to lifetime issues, but if you just want to print it that shouldn't be a problem.
from capnproto-rust.
I'm trying to interface between rust and python with a nice common binary format. Basically I need something that takes &[u8]
and returns the message object (if possible).
I couldn't quite figure out how to get this working with read_message
.
from capnproto-rust.
Basically I need something that takes &[u8] and returns the message object (if possible).
Unfortunately, we don't yet have a function that can conveniently do this without copying the data. We had some discussion and made some progress on this on issue #25.
If you're willing to copy the data, then capnp::serialize::read_message()
should work fine.
from capnproto-rust.
I couldn't quite figure out how to get this working with read_message.
Something like this should work:
fn foo(mut buffer: &[u8]) -> capnp::Result<capnp::OwnedSpaceMessageReader> {
capnp::serialize::read_message(&mut buffer, capnp::ReaderOptions::new())
}
from capnproto-rust.
Oh, I think you mean
capnp::serialize::read_message(&mut Cursor::new(&mut buffer), capnp::ReaderOptions::new())
since read_message
requires a &mut Read
.
Also, is there a way to return a date::Reader
(to go with the OP's example) or at least some sort of less dynamically-typed MessageReader? The only way I've found so far to return a specific type of Reader
, is to take a &mut MallocMessageBuilder
and copy it into there.
Eg:
let message_reader = try!(capnp::serialize_packed::read_message(&mut io::Cursor::new(bytes), capnp::message::DEFAULT_READER_OPTIONS));
let reader : try!(date::Reader = message_reader.get_root());
try!(message_builder.set_root(reader));
Ok(try!(message.get_root::<date::Builder>()).as_reader())
from capnproto-rust.
Also, is there a way to return a date::Reader (to go with the OP's example) or at least some sort of less dynamically-typed MessageReader?
One way to go about this might be to implement a container type, like this
pub struct OwnedMessage<T> {
phantom_data: ::std::marker::PhantomData<T>,
message: ::capnp::OwnedSpaceMessageReader,
}
impl <'a, T> OwnedMessage <T> where T: ::capnp::traits::FromPointerReader<'a> {
pub fn get(&'a self) -> ::capnp::Result<T> {
use capnp::MessageReader;
self.message.get_root()
}
}
You could then return a OwnedMessage<date::Reader>
from foo()
.
from capnproto-rust.
Thanks for the suggestions! Oddly this compiles
fn foo(mut buffer: &[u8]) -> capnp::Result<capnp::OwnedSpaceMessageReader> {
capnp::serialize::read_message(&mut buffer, capnp::ReaderOptions::new())
}
and this does not
fn foo(mut buffer: &[u8]) -> capnp::Result<capnp::OwnedSpaceMessageReader> {
capnp::serialize::read_message(&mut std::io::Cursor::new(&mut buffer), capnp::ReaderOptions::new())
}
with message
cargo build --release (in directory: /home/wayne/projects/rust-py-test/src)
Compiling py-test v0.1.0 (file:///home/wayne/projects/rust-py-test)
lib.rs:63:4: 63:34 error: the trait `std::io::Read` is not implemented for the type `std::io::cursor::Cursor<&mut &[u8]>` [E0277]
lib.rs:63 capnp::serialize::read_message(&mut std::io::Cursor::new(&mut buffer), capnp::ReaderOptions::new())
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I haven't quite figured out this part
You could then return a
OwnedMessage<date::Reader>
fromfoo()
.
I put the code in that you mentioned but can't quite figure out what the body of foo is supposed to be.
pub struct OwnedMessage<T> {
phantom_data: ::std::marker::PhantomData<T>,
message: ::capnp::OwnedSpaceMessageReader,
}
impl <'a, T> OwnedMessage <T> where T: ::capnp::traits::FromPointerReader<'a> {
pub fn get(&'a self) -> ::capnp::Result<T> {
use capnp::MessageReader;
self.message.get_root()
}
}
//fn foo(mut buffer: &[u8]) -> capnp::Result<capnp::OwnedSpaceMessageReader> {
fn foo(mut buffer: &[u8]) -> OwnedMessage<date::Reader> {
// what goes here?
//capnp::serialize::read_message(&mut buffer, capnp::ReaderOptions::new())
}
in the end, i'm hoping to be able to write something like
let my_date = foo(buf);
do_something_with_a_date(&my_date);
from capnproto-rust.
Does this work?
pub struct OwnedMessage<T> {
phantom_data: ::std::marker::PhantomData<T>,
message: ::capnp::OwnedSpaceMessageReader,
}
impl <T> OwnedMessage <T> {
pub fn new(message: ::capnp::OwnedSpaceMessageReader) -> OwnedMessage<T> {
OwnedMessage { phantom_data: ::std::marker::PhantomData, message: message}
}
}
impl <'a, T> OwnedMessage <T> where T: ::capnp::traits::FromPointerReader<'a> {
pub fn get(&'a self) -> ::capnp::Result<T> {
use capnp::MessageReader;
self.message.get_root()
}
}
fn foo(mut buffer: &[u8]) -> ::capnp::Result<OwnedMessage<date::Reader>> {
let message = try!(::capnp::serialize::read_message(&mut buffer, ::capnp::ReaderOptions::new()));
return Ok(OwnedMessage::new(message));
}
from capnproto-rust.
Hm. I think there could be some lifetime issues with what I just suggested. Things will probably work out better if we forget about trying to make OwnedMessage
generic.
pub struct OwnedDateMessage {
message: ::capnp::OwnedSpaceMessageReader,
}
impl OwnedDateMessage {
pub fn get<'a>(&'a self) -> ::capnp::Result<date::Reader<'a>> {
use capnp::MessageReader;
self.message.get_root()
}
}
from capnproto-rust.
(I think my original suggestion with the generic OwnedMessage<T>
needs higher-kinded types to work properly.)
from capnproto-rust.
This is what I'm using now and it compiles/runs:
pub struct OwnedMessage<T> {
message: ::capnp::OwnedSpaceMessageReader,
phantom_data: ::std::marker::PhantomData<T>,
}
impl <'a, T> OwnedMessage <T> where T: ::capnp::traits::FromPointerReader<'a> {
pub fn new(mr: ::capnp::OwnedSpaceMessageReader) -> OwnedMessage<T> {
OwnedMessage {
message: mr,
phantom_data: ::std::marker::PhantomData
}
}
pub fn get(&'a self) -> ::capnp::Result<T> {
use capnp::MessageReader;
self.message.get_root()
}
}
fn read_indir(&self, bref: cafs_capnp::reference::block_ref::Reader) -> Result<OwnedMessage<cafs_capnp::indirect_block::Reader>> {
let indir_bytes = try!(self.read_blockref_vec(bref));
let mut cursor = io::Cursor::new(indir_bytes);
let message_reader = try!(capnp::serialize_packed::read_message(&mut cursor, capnp::message::DEFAULT_READER_OPTIONS));
Ok(OwnedMessage::new(message_reader))
}
Then I'm using it as
let indir = try!(self.read_indir(bref));
let reader = try!(indir.get());
from capnproto-rust.
Your suggestion worked, thanks! Now I just need to figure out how to write out to a &[u8]
let x = foo(buf).unwrap();
let y = x.get().unwrap();
println!("{:?}", y.get_day()); // yayy this works
let new_date = (something about a date builder i think?)
let out_buf &[u8] = new_date.what_is_the_name_of_this_function()
from capnproto-rust.
Thanks for all of your help, my final program is
#[repr(C)]
pub struct bytes_output {
values : *const u8,
len : usize,
}
#[no_mangle]
pub extern fn capnp_ex(external_data: *const u8, data_len : *const size_t) -> bytes_output {
let buf : &[u8] = unsafe{ slice::from_raw_parts(external_data, data_len as usize) };
let x = foo(buf).unwrap();
let y = x.get().unwrap();
println!("capnproto year read! {:?}", y.get_year());
let mut message = MallocMessageBuilder::new_default();
{
let mut out_dt = message.init_root::<date::Builder>();
out_dt.set_year(y.get_year() + 1);
out_dt.set_month(1);
out_dt.set_day(1);
}
let mut out_buf : Vec<u8> = Vec::new();
capnp::serialize::write_message( &mut out_buf, &message );
bytes_output {
values : out_buf.as_ptr(),
len : out_buf.len(),
}
}
and the python that calls it
import cffi
import capnp
#setup ffi
ffi = cffi.FFI()
ffi.cdef('''
typedef struct {
char* values;
size_t len;
} bytes_output;
bytes_output capnp_ex(char*, size_t);
''')
lib = ffi.dlopen("./target/release/libtest.so")
# setup capnp
capnp.remove_import_hook()
example = capnp.load('schema/example.capnp')
mydate = example.Date.new_message(year=2015, month=5, day=31)
mydatebytes = mydate.to_bytes()
myresp = lib.capnp_ex(ffi.new('char[]', mydatebytes), len(mydatebytes))
def out_iter(resp):
for i in range(resp.len):
yield resp.values[i]
print(myresp)
with open('tmp.bin','wb') as outf:
for byte in out_iter(myresp):
outf.write(byte) # must be a better way to do this..
with open('tmp.bin', 'rb') as inf:
myrespdate = example.Date.read( inf )
print(myrespdate)
for some reason the python capnp implementation relies on files quite a bit and I couldn't find a way around using files.
from capnproto-rust.
Related Issues (20)
- `capnp_futures::serialize_packed::try_read_message()` is blocked forever with 100% cpu usage when the input stream is closed
- Backport capnp-futures changes from master to 0.14 version HOT 2
- no_std, no alloc serialization HOT 10
- Invoking closure on the event loop to get a mutable reference to self HOT 1
- How to read unknown number of messages HOT 2
- Serializing message to &[u8] HOT 2
- Improve performance of Reader get_named HOT 4
- Primitive list panicks if the list type is `Void` HOT 1
- Null values for non active union members HOT 3
- Switch Allocator to act more like an owned arena HOT 8
- Reader fails with out of bounds pointer error when reading from compressed stream HOT 1
- Revert text setter ergonomics pessification; add raw_set methods HOT 6
- calculator example fails with java server: Cannot read capability pointer without capTable HOT 1
- c++.capnp compiles to unusable module name HOT 6
- How to send generated readers across threads HOT 3
- From bytes to TypedReader HOT 6
- Allowing construction of NoAllocSliceSegments with buffer and NoAllocBufferSegmentType HOT 7
- Kudos to builders of this Rust library - Every single one of you. HOT 3
- [Question]How to extract generic type values HOT 5
- capnp_rpc does not work on monoio HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from capnproto-rust.