Design: boxed preads about scroll HOT 6 OPEN

Do you have a real world example that would use this?

from scroll.

Yes, similar to the example above, I was dealing with some C structs which had some header information, etc., then a large byte array containing image bytes, then some more stuff.
I wanted to pread the struct, match on some header values, validate, then do something with the bytes, then write them out to disk with slightly different header values.
Scroll would have been extremely fast way to prototype/work with the data, but because I couldn't box the pread data, it exploded my stack; testing with the pwrite version taking references, I was able to write the data, which was one half of the problem.
Hope that's clear?
I think this will generally come up in FFI with large C structs of byte arrays where the api expects a malloc(sizeof(LargeStruct)) to construct the initial type.

from scroll.

I was hoping for an actual real world example. It seems to me that large fixed size arrays are quite uncommon, and even then you can usually use Vec<u8> anyway. So the use case is:

• you have a struct containing a large fixed size array
• you need to pass this struct to C with FFI so you can't use Vec?

from scroll.

It was the other way around, basically:

struct Foo {
  struct Header header;
  uint8_t data [0x8000];
  struct Footer footer; 
}

In rust land I then wanted to:

let x = bytes.pread::<Foo>(0)?; // this fails because we allocate x on the stack.
// check x.header, do stuff, manipulate x.bytes, pwrite

from scroll.

So I had something like this in mind 😈 , but it still overflows the stack (it also allocates twice which sucks):

impl<'a, Ctx, T> TryFromCtx<'a, Ctx> for Box<T>
where
    T: TryFromCtx<'a, Ctx>,
    Ctx: Copy,
    error::Error: From<<T as TryFromCtx<'a, Ctx>>::Error>
{
    type Error = error::Error;
    #[inline]
    fn try_from_ctx(src: &'a [u8], ctx: Ctx) -> result::Result<(Self, usize), Self::Error> {
        let res = Box::new(TryFromCtx::try_from_ctx(src, ctx)?);
        Ok((Box::new(res.0), res.1))
    }
}

#[test]
fn pwrite_big_struct() {
    use scroll::{LE, Pwrite};
    #[derive(Pread, Pwrite, Clone)]
    struct Big {
        arr: [u8; 10000000],
    }
    let mut bytes = vec![0; 10000001];
    let big = bytes.pread_with::<Box<Big>>(0, LE).unwrap();
    let _ = bytes.pwrite_with(big.as_ref(), 0, LE).unwrap();
    assert!(false);
}

I'm wondering if its even possible to write in scroll TryFromCtx as is without blowing up the stack?

from scroll.

If Big::try_from_ctx(src, ctx) overflows the stack, then wrapping it in Box::new won't change anything. You need placement new or an equivalent (pass in a preallocated pointer for the destination of the read, instead of returning a value), which is incompatible with returning by value.

from scroll.