Paramaterize name in message-store-level.ts

Decentralized Web Node (DWN) SDK

Introduction
Installation
Additional Steps
Usage
Release/Build Process
- Stable Build
- Unstable Build
Some projects that use this library:
Architecture
Project Resources

Introduction

This repository contains a reference implementation of Decentralized Web Node (DWN) as per the specification. This specification is in a draft state and very much so a WIP. For the foreseeable future, a lot of the work on DWN will be split across this repo and the repo that houses the specification. The current implementation does not include all interfaces described in the DWN spec, but is enough to begin building test applications.

This project is used as a dependency by several other projects.

Proposals and issues for the specification itself should be submitted as pull requests to the spec repo.

Running online environment

Interested in contributing instantly? You can make your updates directly without cloning in the running CodeSandbox environment.

Installation

If you are interested in using DWNs and web5 in your web app, you probably want to look at web5-js, instead of this repository. Head on over here: https://github.com/TBD54566975/web5-js.

For advanced users wishing to use this repo directly:

npm install @tbd54566975/dwn-sdk-js

Additional Steps

This package has dependency on @noble/ed25519 and @noble/secp256k1 v2, additional steps are needed for some environments:

Node.js <= 18

// node.js 18 and earlier,  needs globalThis.crypto polyfill
import { webcrypto } from "node:crypto";
// @ts-ignore
if (!globalThis.crypto) globalThis.crypto = webcrypto;

React Native

Usage of DWN SDK in react native requires a bit of set up at the moment. To simplify, we've published an npm package that can be used to set everything up. Follow the instructions to get started.

Usage in Browser:

dwn-sdk-js requires 2 polyfills: crypto and stream. we recommend using crypto-browserify and stream-browserify. Both of these polyfills can be installed using npm. e.g. npm install --save crypto-browserify stream-browserify

Vanilla HTML / JS

DWN SDK includes a polyfilled distribution that can imported in a module script tag. e.g.

<!DOCTYPE html>
<html lang="en">
<body>
  <script type="module">
     // Import necessary modules from external sources using ES6 modules.
    import { Dwn, DataStream, DidKeyResolver, Jws, RecordsWrite } from 'https://cdn.jsdelivr.net/npm/@tbd54566975/[email protected]/dist/bundles/dwn.js'
    import { MessageStoreLevel, DataStoreLevel, EventLogLevel } from 'https://cdn.jsdelivr.net/npm/@tbd54566975/[email protected]/dist/bundles/level-stores.js'

    // Create instances of various components from the imported modules.
    const messageStore = new MessageStoreLevel();
    const dataStore = new DataStoreLevel();
    const eventLog = new EventLogLevel();
    const dwn = await Dwn.create({ messageStore, dataStore, eventLog });

    // Generate a did:key DID (Decentralized Identifier).
    const didKey = await TestDataGenerator.generateDidKeyPersona();

    // Create some data to be stored.
    const encoder = new TextEncoder();
    const data = encoder.encode('Hello, World!');

    // Create a RecordsWrite message to be stored in DWN.
    const recordsWrite = await RecordsWrite.create({
      data,
      dataFormat: 'application/json',
      published: true,
      schema: 'yeeter/post',  // Specify a schema for the data.
      signer: Jws.createSigner(didKey) // Sign the data using the generated DID key.
    });

    // Create a readable stream from the data to be stored.
    const dataStream = DataStream.fromBytes(data);
    // Process the RecordsWrite message using the DWN instance.
    const result = await dwn.processMessage(didKey.did, recordsWrite.message, { dataStream });

    // Log the processing result status and perform an assertion.
    console.log(result.status);
    console.assert(result.status.code === 202)

    // Close the DWN instance, freeing up resources.
    await dwn.close()

  </script>
</body>

</html>

Webpack >= 5

Add the following to the top level of your webpack config (webpack.config.js)

resolve: {
  fallback: {
    stream: require.resolve("stream-browserify"),
    crypto: require.resolve("crypto-browserify")
  }
}

Vite

Add the following to the top level of your vite config (vite.config.js)

define: {
  global: 'globalThis'
},
resolve: {
  alias: {
    'crypto': 'crypto-browserify',
    'stream': 'stream-browserify'
  }
}

esbuild

We recommend using node-stdlib-browser instead of crypto-browserify and stream-browserify individually. Example usage:

import esbuild from 'esbuild'
import stdLibBrowser from 'node-stdlib-browser'
import polyfillProviderPlugin from 'node-stdlib-browser/helpers/esbuild/plugin'

import { createRequire } from 'node:module';

const require = createRequire(import.meta.url);

// Build the project using esbuild.
esbuild.build({
  entryPoints: ['dwn-sdk-test.js'],
  platform: 'browser',
  bundle: true,
  format: 'esm',
  outfile: 'dist/dwn-sdk-test.js',

  // Inject the specified shim for Node.js standard library browser compatibility.
  inject      : [require.resolve('node-stdlib-browser/helpers/esbuild/shim')],
  // Use the polyfillProviderPlugin to provide polyfills for Node.js standard library.
  plugins     : [polyfillProviderPlugin(stdLibBrowser)],
  // Define 'global' as 'globalThis' to ensure compatibility with global objects.
  define      : {
    'global': 'globalThis'
  }
})

Usage

API docs

import { Dwn, DataStream, DidKeyResolver, Jws, RecordsWrite } from '@tbd54566975/dwn-sdk-js';
import { DataStoreLevel, EventLogLevel, MessageStoreLevel } from '@tbd54566975/dwn-sdk-js/stores';

// Initialize the required stores and components for the DWN SDK.
const messageStore = new MessageStoreLevel();
const dataStore = new DataStoreLevel();
const eventLog = new EventLogLevel();
const dwn = await Dwn.create({ messageStore, dataStore, eventLog });

// Generate a did:key DID (Decentralized Identifier).
const didKey = await TestDataGenerator.generateDidKeyPersona();

// Create some data to be stored.
const encoder = new TextEncoder();
const data = encoder.encode('Hello, World!');

// Create a RecordsWrite message to be stored in the DWN.
const recordsWrite = await RecordsWrite.create({
  data,
  dataFormat: 'application/json',
  published: true, // Mark the data as published.
  schema: 'yeeter/post', // Specify a schema for the data.
  signer: Jws.createSigner(didKey) // Sign the data using the generated DID key.
});

// Create a readable stream from the data to be stored.
const dataStream = DataStream.fromBytes(data);
// Process the RecordsWrite message using the DWN instance.
const result = await dwn.processMessage(didKey.did, recordsWrite.message, { dataStream });
// Log the processing result status.
console.log(result.status);

With a web wallet installed:

const result = await window.web5.dwn.processMessage({
  method: "RecordsQuery",
  message: {
    filter: {
      schema: "http://some-schema-registry.org/todo",
    },
    dateSort: "createdAscending",
  },
});

Custom Tenant Gating

By default, all DIDs are allowed as tenants. A custom tenant gate implementation can be provided when initializing the DWN.

import { ActiveTenantCheckResult, Dwn, TenantGate, DataStoreLevel, EventLogLevel, MessageStoreLevel } from '@tbd54566975/dwn-sdk-js';

// Define a custom implementation of the TenantGate interface.
class CustomTenantGate implements TenantGate {
  public async isActiveTenant(did): Promise<ActiveTenantCheckResult> {
    // Custom implementation
  }
}

// Initialize the required stores and components for the DWN SDK.
const messageStore = new MessageStoreLevel();
const dataStore = new DataStoreLevel();
const eventLog = new EventLogLevel();
// Create an instance of the custom TenantGate.
const tenantGate = new CustomTenantGate();
// Create a DWN instance with configured stores, logs, and the custom TenantGate.
const dwn = await Dwn.create({ messageStore, dataStore, eventLog, tenantGate });

Custom Signature Signer

If you have the private key readily available, it is recommended to use the built-in PrivateKeySigner. Otherwise, you can implement a customer signer to interface with external signing service, API, HSM, TPM etc and use it for signing your DWN messages:

// Create a custom signer implementing the Signer interface.
class CustomSigner implements Signer {
  public keyId = 'did:example:alice#key1'; // Specify the key ID.
  public algorithm = 'EdDSA'; // Specify the signing algorithm (valid `alg` value published).
  https://www.iana.org/assignments/jose/jose.xhtml
  public async sign (content: Uint8Array): Promise<Uint8Array> {
    ... // custom signing logic
  }
}

// Create an instance of the custom signer for authorization.
const signer = new CustomSigner();

// Define options for creating a RecordsWrite message.
const options: RecordsWriteOptions = {
  ...
  signer // Use the custom signer for authorization.
};

// Create a RecordsWrite message with the specified options.
const recordsWrite = await RecordsWrite.create(options);

Release/Build Process

The DWN JS SDK releases builds to npmjs.com. There are two build types: stable build and unstable build.

Stable Build

This is triggered manually by:

Increment version in package.json in Semantic Versioning (semver) format.
Merge the change into main branch
Create a release from GitHub.

An official build with version matching the package.json will be published to npmjs.com.

Unstable Build

Every push to the main branch will automatically trigger an unstable build to npmjs.com for developers to experiment and test.

The version string contains the date as well as the commit hash of the last change.

An example version string:

0.0.26-unstable-2023-03-16-36ec2ce

0.0.26 came from version in package.json
2023-03-16 indicates the date of March 16th 2023
36ec2ce is the commit hash of the last change

Some projects that use this library:

Architecture

NOTE: The diagram is a conceptual view of the architecture, the actual component abstraction and names in source file may differ.

Project Resources

Resource	Description
CODEOWNERS	Outlines the project lead(s)
CODE_OF_CONDUCT.md	Expected behavior for project contributors, promoting a welcoming environment
CONTRIBUTING.md	Developer guide to build, test, run, access CI, chat, discuss, file issues
GOVERNANCE.md	Project governance
LICENSE	Apache License, Version 2.0
Q_AND_A.md	Questions and answers on DWN

	async open(): Promise<void> {
	await this.db.open();

	// TODO: look into using the same level we're using for blockstore
	// TODO: parameterize `name`
	// calling `searchIndex` twice causes the process to hang, so check to see if the index
	// has already been "opened" before opening it again.
	if (!this.index) {
	this.index = await searchIndex({ name: this.config.indexLocation });
	}
	}

	export class MessageStoreLevel implements MessageStore {
	config: MessageStoreLevelConfig;
	db: BlockstoreLevel;
	// levelDB doesn't natively provide the querying capabilities needed for DWN. To accommodate, we're leveraging
	// a level-backed inverted index
	// TODO: search-index lib does not import type `SearchIndex`. find a workaround
	index;


	// TODO: replace with JSON Schema based validation
	// more info about the `JsonWebKey2020` type can be found here:
	// https://www.w3.org/TR/did-spec-registries/#jsonwebkey2020
	if (verificationMethod.type !== 'JsonWebKey2020') {
	throw new Error(`verification method [${kid}] must be JsonWebKey2020`);
	}

	/**
	* TODO: add docs
	* @param message
	*/
	async processMessage(rawMessage: object, ctx: Context): Promise<MessageReply> {
	let message: MessageSchema;

	if (!isValid) {
	// TODO: build helpful errors object using returned errors
	// Every time a validation function is called the errors property is overwritten.
	// const errors = [...validateFn.errors];
	throw new Error('Invalid message.');
	}
	}

	const { publicKeyJwk: publicJwk } = verificationMethod;

	// TODO: replace with JSON Schema based validation
	// more info about the `publicJwk` property can be found here:
	// https://www.w3.org/TR/did-spec-registries/#publicJwk
	if (!publicJwk) {
	throw new Error(`publicKeyJwk property not found on verification method [${kid}]`);
	}

	return publicJwk as PublicJwk;
	}

	// TODO: change type of `query`
	query(query: any, ctx: Context): Promise<MessageSchema[]>;

	// TODO: replace with JSON Schema based validation
	// more info about the `publicKeyJwk` property can be found here:
	// https://www.w3.org/TR/did-spec-registries/#publickeyjwk
	if (!publicKeyJwk) {
	throw new Error(`publicKeyJwk property not found on verification method [${kid}]`);
	}

	const indexDocument: any = { _id: encodedBlock.cid.toString(), method, objectId };

	// TODO: clean this up and likely move it elsewhere (e.g. a different function) so that it can be used elsewhere
	if (descriptor.method === 'PermissionsRequest') {
	indexDocument.ability = descriptor.ability;
	indexDocument.requester = descriptor.requester;
	}

	await this.index.PUT([indexDocument]);

	// TODO: figure out if we need to support ALL verification method properties
	// listed here: https://www.w3.org/TR/did-spec-registries/#verification-method-properties
	export type VerificationMethod = {
	id: string
	// one of the valid verification method types as per
	// https://www.w3.org/TR/did-spec-registries/#verification-method-types
	type: string
	// DID of the key's controller
	controller: string
	// a JSON Web Key that conforms to https://datatracker.ietf.org/doc/html/rfc7517
	publicKeyJwk?: PublicJwk
	// a string representation of
	// https://datatracker.ietf.org/doc/html/draft-multiformats-multibase-05
	publicKeyMultibase?: string
	};

	public async resolve(did: string): Promise<DIDResolutionResult> {
	// naively validate requester DID
	// TODO: add better DID validation
	const splitDID = did.split(':', 3);
	if (splitDID.length < 3) {
	throw new Error(`${did} is not a valid DID`);
	}

	public async resolve(did: string): Promise<DIDResolutionResult> {
	// naively validate requester DID
	validateDID(did);
	const splitDID = did.split(':', 3);

	/**
	* @param did - the DID to validate
	*/
	export function validateDID(did: unknown): void {
	// @see https://www.w3.org/TR/2021/PR-did-core-20210803/#did-syntax
	const DID_REGEX = /^did:[a-zA-Z0-9.!#$%&'+\/=?^_`{\|}~-]+:[a-zA-Z0-9.!#$%&'+\/=?^_`{\|}~-]+$/;

	if (typeof did !== 'string') {
	throw new TypeError(`DID is not string: ${did}`);
	}

	if (!DID_REGEX.test(did)) {
	throw new TypeError(`DID is not a valid DID: ${did}`);
	}
	}

	// TODO: add DIDCache to constructor method signature
	constructor(resolvers: DIDMethodResolver[]) {
	this.didResolvers = new Map();

	for (const resolver of resolvers) {
	this.didResolvers.set(resolver.method(), resolver);
	}
	}

	/**
	* TODO: add docs
	*/
	export class DIDResolver {
	didResolvers: Map<string, DIDMethodResolver>;


	/**
	* TODO: add JSDoc
	* @param schemaName
	* @param payload
	* @returns
	*/
	export function validate(schemaName: string, payload: any): void {

	// TODO: add logic to prevent validating duplicate signatures
	export class GeneralJwsVerifier {
	jws: GeneralJws;

	constructor(jws: GeneralJws) {
	this.jws = jws;
	}

	async verify(didResolver: DIDResolver): Promise<VerificationResult> {
	const signers: string[] = [];

	for (const signature of this.jws.signatures) {
	const protectedBytes = base64url.baseDecode(signature.protected);
	const protectedJson = new TextDecoder().decode(protectedBytes);

	const { kid } = JSON.parse(protectedJson);
	const did = GeneralJwsVerifier.extractDid(kid);
	const publicJwk = await GeneralJwsVerifier.getPublicKey(did, kid, didResolver);

	const isVerified = await GeneralJwsVerifier.verifySignature(this.jws.payload, signature, publicJwk);

	if (isVerified) {
	signers.push(did);
	} else {
	throw new Error(`signature verification failed for ${did}`);
	}
	}

	return { signers };
	}

	// TODO: should we add an explicit check to ensure that there's only 1 signer?
	const { signers } = await request.verifyAuth(didResolver);
	const [ signer ] = signers;

	if (signer !== request.grantedTo) {
	throw new Error('grantee must be signer');
	}

	throw new Error(`publicKeyJwk property not found on verification method [${kid}]`);
	}

	// TODO: figure out if we need to check to ensure that `controller` === did in kid
	// are the same. This may matter more for a `PermissionsRequest`

tbd54566975 / dwn-sdk-js Goto Github PK

dwn-sdk-js's Introduction