Right now there are two different worlds for consolidated vs non-consolidated metadata

Added in <a class="issue-link js-issue-link" data-error-text="Failed to load title" da

Unifying consolidated and non-consolidated interfaces about zarrita.js HOT 3 CLOSED

manzt commented on September 6, 2024

Unifying consolidated and non-consolidated interfaces

from zarrita.js.

Comments (3)

manzt commented on September 6, 2024

Another idea is that this could just be a special store (wrapper):

interface Listable {
  contents(): Array<{ path: AbsolutePath, kind: "array" | "group" }>;
}

async function withConsolidated<Store extends Readable>(store: Store) Pick<Store, "get"> & Listable {
  const known_metadata = await try_consolidated(store);
  return {
    get(...args: Parameters<Store["get"]>) {
      let [key, opts] = args;
      if (key in known_metadata) return known_metadata[key];
      let maybe_bytes = await store.get(key, opts);
      if (is_meta_key(key) && maybe_bytes) { // add to known_metadata
      return maybe_bytes;
    },
    contents() {
      return list_nodes(known_metadata);
    }
  }
}

The reason I chose to use contents over keys is because keys for Map would list all contents (including chunks).

Then openConsolidated could wrap withConslidated. Wonder what you think @keller-mark.

import { withConsolidated, FetchStore, open } from "zarrita";

let store = await withConsolidated(new FetchStore("http://localhost:8080"));
let contents = store.contents() // [ {path: "/", kind: "group" }, { path: "/foo", kind: "array" }, ...]
let foo = await open(contents[1].path, { kind: "array" });