Oh hi! I hadn't seen this until I saw it pop up in rust-lang/rust#97052.
I'm part of alloc-wg and about two months ago I picked up the Storage API again in this repo: https://github.com/CAD97/storages-api (docs: https://cad97.github.io/storages-api/storage_api/index.html)
As part of this refinement and simplification push, I ended up converting the Storage
trait to be untyped: that is, Storage::Handle
is no longer a GAT, and notably this means that handle resolution can return regular references to [MaybeUninit<u8>]
rather than pointers with difficult to explain invalidation rules.
There's some miscellaneous renaming involved, but also the simplification brought us down to a set of just four traits: Storage
, SharedMutabilityStorage
, PinningStorage
, and MultipleStorage
. I've been advised (and agree) that controlling complexity is important here for a path to std inclusion; even with this simplified API surface without GAT, it's still a contender for the most complex API exposed by std. And that complexity is safety critical.
I'm fairly convinced that the current shape in my repo is minimal for what we want to support, but I wasn't thinking the previous POC was terribly over-engineered1 either, so it's possible that a further simplification is hiding.
matthieu-m agrees that the untyping of storage is probably a good direction; the previous storage API was not typed due to some conscious decision due to necessity, but because it evolved from C++'s std::allocator
design, which is typed, and matthiue-m just hadn't considered using an untyped interface. This does mean that usage of the storage traits becomes more unsafe
/difficult, but this is easily addressed by adding a typed RawBox
around Storage
, RawVec
around Storage
of slices, and maybe RawArena
around MultipleStorage
? as a bridge between raw storage and high-level collections.
(It's also quite possible that matthieu-m overlooked the possibility of making handles untyped because of the relative newness of the pointee metadata APIs at the time, in order to draw the conclusion that S::Handle<T>
can be replaced by (S::Handle, <T as Pointee>::Metadata)
.)
Anyway, getting to the point:
I received a Rust Foundation Community Grant. My project is sort of two projects; the first is unrelated work in the tracing ecosystem, but the second half (scheduled Oct..=Dec) is to develop and push the Storage API to an MCP. rust-lang/rust#97052 is actually an early start on prerequisite work for upstreaming storages, as otherwise a storage-generic Box
can't be coerced.
pub struct RawBox<T: ?Sized, S: Storage> {
handle: S::Handle,
metadata: <T as Pointee>::Metadata, // -> JustMetadata<T>
storage: S,
}
My plan for MCPing the Storage API is basically broken down into the following three parallel steps:
- Recruit a T-lang member to liason/proxy/sponsor/whatever-the-word-is the proposal.
- Provide a proof of concept implementation for minimally all currently allocator-generic std collections on top of the Storage API.
- I'm still up-in-the-air about whether to do this in-tree on the std impls directly in a fork, or to reimplement them in a separate repo (like you've done here).
- Part of what makes this interesting is that this makes std collections
#[no_std]
compatible, so while refactoring in-place has the benefits of full API coverage and ease of "build with my fork of std" testing, wanting to move things into libcore means the diff would be little better than an external crate. Plus having to deal with std iteration times.
- Write the MCP.
- Document the supported use cases / design constraints / etc.
- Document why the API has the shape it does.
- My intent here is to guide the reader through inventing the storage traits themselves, like the "reinventing
tower::Service
" article.
- Make the argument for putting this into std in addition to the current unstable
Allocator
trait.
Ultimately, I think that this MCP is where the Storage API either sinks or swims. Either it gets accepted for in-tree implementation/experimentation, or it gets rejected, likely on the matter of being insufficiently motivated for std inclusion given its complexity. Especially given the async-wg group w.r.t. async in traits (e.g. dyn*
), now really does feel like the time to say if we want to make Box
the "owning handle to some storage" or if it's just an owning pointer to allocated memory. My claim is that Box<T, ?>
can be both &move T
and dyn* T
, as well as InlineMaxSizeDst<T>
and so much more, so its a matter of proving it worth it.
I'd like to work together, or at the very least avoid duplicating work you've already done unnecessarily. Adding your experience working with (your modified copy of) the storage API will help to strengthen the motivation provided in the MCP.
...I don't really know how to end this info dump. I guess...
TL;DR: Hi ๐ I'm working on the Storage API and would like to incorporate your experience working with it.