Giter VIP home page Giter VIP logo

Comments (6)

rhashimoto avatar rhashimoto commented on May 24, 2024 2

@Azarattum Just skimming it, it looks reasonable. You might not need locking if you don't have multiple connections accessing the database concurrently. I write very little for node so I don't know anything about locking or bundling there.

This is getting off-topic for this issue. If you have other questions I'd suggest moving to a new Discussion so people subscribed here don't get notifications.

from wa-sqlite.

rhashimoto avatar rhashimoto commented on May 24, 2024

Added to FAQ. Thanks!

from wa-sqlite.

Azarattum avatar Azarattum commented on May 24, 2024

Is there a VFS adapter that supports native filesystem with node:fs?

from wa-sqlite.

rhashimoto avatar rhashimoto commented on May 24, 2024

Is there a VFS adapter that supports native filesystem with node:fs?

@Azarattum I have not written one. My interests are mainly in the browser, and I know there is already a way to use a native SQLite database from node (though I have never tried it myself) so it's not something I care much about.

It probably wouldn't be difficult for you to write one, but my guess is that a node extension calling a native SQLite library is going to be measurably faster than a WASM SQLite calling back to Javascript calling node:fs.

from wa-sqlite.

Azarattum avatar Azarattum commented on May 24, 2024

@rhashimoto I've actually written one in the meantime. What do you think, does it look OKish? I wasn't sure how to implement locks though... I wanted to use wa-sqlite over native bindings to achieve cross-platform portability. Ideally i would want my app to compile into a single .cjs file that can be ran be Node. Do you have any idea on how that might be done? I'm using vite to bundle my project. However, it doesn't play with webassembly quite well...

node-vfs.ts
// @ts-ignore
import { access, FileHandle, open, rm } from "node:fs/promises";
import {
  SQLITE_OPEN_DELETEONCLOSE,
  SQLITE_IOERR_SHORT_READ,
  SQLITE_OPEN_CREATE,
  SQLITE_CANTOPEN,
  SQLITE_IOERR,
  SQLITE_OK,
  Base,
  // @ts-ignore
} from "@vlcn.io/wa-sqlite/src/VFS";
// @ts-ignore
import { resolve } from "node:path";

type File = { handle: FileHandle; flags: number; path: string };
type Result<T = number> = { set: (value: T) => void };
type Data = { size: number; value: Int8Array };

export class NodeVFS extends Base {
  files = new Map<number, File>();
  name = "node";

  xOpen(path: string | null, id: number, flags: number, outFlags: Result) {
    return this.handle(null, async () => {
      path =
        path ||
        Math.floor(Math.random() * Number.MAX_SAFE_INTEGER).toString(36);
      path = resolve(path) as string;

      let mode = "r+";
      try {
        await access(path);
      } catch (error) {
        if (flags & SQLITE_OPEN_CREATE) mode = "w+";
        else throw error;
      }

      try {
        const handle = await open(path, mode);
        this.files.set(id, { handle, flags, path });
        outFlags.set(flags);
      } catch {
        return SQLITE_CANTOPEN;
      }
    });
  }

  xClose(id: number) {
    return this.handle(id, async ({ handle, flags, path }) => {
      this.files.delete(id);
      await handle.close();
      if (flags & SQLITE_OPEN_DELETEONCLOSE) await rm(path);
    });
  }

  xRead(id: number, data: Data, offset: number) {
    return this.handle(id, async ({ handle }) => {
      const size = (await handle.stat()).size;
      const start = Math.min(offset, size);
      const end = Math.min(offset + data.size, size);
      const length = end - start;

      if (length) {
        // @ts-ignore
        const buffer = Buffer.alloc(length);
        await handle.read(buffer, 0, length, start);
        data.value.set(buffer);
      }

      if (length < data.size) {
        data.value.fill(0, end);
        return SQLITE_IOERR_SHORT_READ;
      }
    });
  }

  xWrite(id: number, data: Data, offset: number) {
    return this.handle(id, async ({ handle }) => {
      await handle.write(new Uint8Array(data.value), 0, data.size, offset);
    });
  }

  xTruncate(id: number, size: number) {
    return this.handle(id, async ({ handle }) => {
      await handle.truncate(size);
    });
  }

  xFileSize(id: number, outSize: Result) {
    return this.handle(id, async ({ handle }) => {
      outSize.set((await handle.stat()).size);
    });
  }

  xDelete(path: string) {
    return this.handle(null, async () => {
      await rm(path);
    });
  }

  xAccess(path: string, flags: number, accessOut: Result) {
    return this.handle(null, async () => {
      try {
        await access(path, flags);
        accessOut.set(1);
      } catch {
        accessOut.set(0);
      }
    });
  }

  handle<T extends number | null>(
    id: T,
    fn: (file: T extends number ? File : undefined) => Promise<void | number>
  ) {
    return (this as any).handleAsync(() => {
      const file = id ? this.files.get(id) : undefined;
      if (id && !file) return Promise.resolve(SQLITE_IOERR);
      return fn(file as any)
        .then((code) => code || SQLITE_OK)
        .catch(() => SQLITE_IOERR);
    });
  }
}

from wa-sqlite.

rhashimoto avatar rhashimoto commented on May 24, 2024

I changed the emscripten ENVIRONMENT option to "web,worker" to address reported issues with webpack. This may unfortunately cause the .mjs files not to load properly in node. If that happens the workaround should be to use a custom build with "node" added to ENVIRONMENT.

I apologize if this does break things for node users but there are more webpack users.

from wa-sqlite.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.