manzt / zarrita.js Goto Github PK

Adapted from: gzuidhof/zarr.js#69

A KeyError is a built in python error that's thrown when a key is missing from a MutableMapping. In JavaScript, the equivalent behavior is to just return undefined when a key is missing. Making this change would remove a lot of the try/catch blocks from the core implementation and instead we would just call:

// Store returns 'undefined' instead of throwing custom error if chunk is missing.
// Otherwise the store can throw any other error and it will propagate up.
const chunk = await this.store.get(ckey);
if (!chunk) {
  /* create the missing chunk */ 
} else {
  /* decode the chunk */
}

I introduced ZarrArray.do to allow tree-shaking of class methods:

import { v2, get, set } from 'zarrita';

const a = await v2.get_hierarchy(...).get_array('/');
a.do(get, null) // set can be tree-shaken as it is not used

However, the inferred return types from .do aren't very precise, and the alternative of just passing ZarrArray as a parameter to get/set is much more simple:

import { v2, get, set } from 'zarrita';

const a = await v2.get_hierarchy(...).get_array('/');
get(a, null) // set can be tree-shaken as it is not used

I implemented optional float16 support in zarr.js recently. I believe the same strategy should work here, but I'm not sure how type inference will work.

I tried porting some of the strided array set functionality to zarr-lite and seem to be getting errors for slicing (empty arrays), specifically when the output is a lower dimension than the chunk shape.

I should come up with some separate tests for src/opts.js to test various slicing and indexing. Just need to find some time...

Rather than returning Promise<Uint8Array> , we could unify the store interface around Web-standard Response to expose additional features to zarr.Array instances.

Changes:

• Use the Response.ok or response.status to determine if the item is in the store or not (rather than Uint8Array | undefined).
• Exposes conveniences to read response directly via .json(), .text(), .arrayBuffer(), and .blob(), rather than copying array buffers etc.
• Ability process Response.body as a stream. This is the most interesting/exciting to me, but it's not clear if it would be useful. Ideally there is a world were chunks could be partially decoded, etc.

Rather than handling Responses within zarr.Array, we could look into something like a StorageTransformer layer, which by default just reads the response in full.

• zarr-developers/zarr-specs#134

It's a little clunky to use ops methods because they require a separate import. For those concerned with bundle size, this is fine, but it might be unintuitive to a new user. As an alternative, we could export the builtin ops and add a namespace for ndarray:

Current

import * as zarr from "zarrita/v2";
import { get, set } from "zarrita/ops";
import * as ndops from "zarrita/ndarray";
import FetchStore from "zarrita/storage/fetch";

let root = await zarr.get(new FetchStore("http://localhost:8080/data.zarr");
let res = get(root);
set(root, ...);
ndops.get(root);
ndopts.set(root, ...);

Proposed

import * as zarr from "zarrita/v2";
import FetchStore from "zarrita/storage/fetch";

let root = await zarr.get_node(new FetchStore("http://localhost:8080/data.zarr")); // or `zarr.open` ?
let res = zarr.get(root);
zarr.set(root, ...);
zarr.ndarray.get(root);
zarr.ndarray.set(root, ...);

The primary issue I see is that by default import * as zarr imports both ops and would likely increase bundle size on average. However, for those concerned with bundle size, named imports will remain treeshakeable.

import { get_node, get } from "zarrita/v2";
import FetchStore from "zarrita/storage/fetch";

let root = await get_node(new FetchStore("http://localhost:8080/data.zarr"));
let res = get(root);

Similar to zarr.js, rather than returning undefined to indicate a missing key, maybe explore throwing a an error with a literal __zarr__ key.

start, end, step cannot be floating numbers. Using type-fest's Integer type would allow enforcing this constraint at the type-level: https://github.com/sindresorhus/type-fest/blob/main/source/numeric.d.ts