Background

Currently @faceteer/facet expects a provisioned instance of Dynamo DB from the aws-sdk.

This causes issues when trying to load the library in non-node environments, like Cloudflare Workers.

Removing the requirement on aws-sdk could make it easier to deploy to multiple places.

Proposal

We use https://github.com/mhart/aws4fetch, and update all of our calls to hit the DynamoDB API directly.

There are many use cases for DynamoDB Transactions, we should add support for this.

When performing deletes, it is useful to be able to differentiate between "this item existed and was deleted" and "this did not exist, and there was nothing to delete". For example, a RESTful DELETE endpoint can return a 404 or 200 based on whether the item already existed or not.

Right now the only way to differentiate between the two is to first perform a lookup then delete, which is not even necessarily accurate since there can be a data race between the two operations.

A better way to do this might be to use the attribute_exists condition with the PK attribute to make the delete fail if the item does not exist.

What

Create a script which can be used to iterate through every record in DynamoDB and migrate the data structure. Perhaps this needs its own repository.

Delete

We need to add a delete function to our facets.

This function should take an individual item, or an array of items to delete.

If we get an array of items we should use the bulk delete endpoint and iterate through until everything is deleted.

If we get a single item we should allow for a conditional deletes.

Looking at the current implementation for put will be a good idea.

What

Add support for updating an item within a Facet with update(). This behavior would work similarly as put(), except not all attributes are required.

Allows each record to be aware of what facet it belongs to.

This is useful when subscribed to a DynamoDB Stream or scanning through records with a script.

Consider a case where you need to bulk delete items from a facet. You can't perform deletes on a query, so you need to instead query for the items you want to delete, then issue deletes on them.

But one problem emerges: say your database includes items that were false and can no longer be validated. You can't query for these items as they will fail the validation, which means you then can't delete them.

A good solution to this problem would be to temporarily override the facet validator on a query. The override could look like this:

const items = await MyFacet.query({ someId }).withValidator((item) => 'someId' in item).list();
MyFacet.delete(items);

The same partition key can have many different entities inside of it.

To really utilize DynamoDB Faceteer should be able to support this use case.

If a facet is configured to use shards, the default behavior should be to query every shard if none is specified.

Some positive outcomes:

• Minimizes code when querying all shards (as shown below)
• Removes anti-pattern to define a shard count in some configuration file to reference outside of facet
• Allows previous pattern to prevent breaking changes

Before

const promises = [];
for(let i = 0; i < SHARD_COUNT; i++) {
    promises.push(Facet.query({ partition }, i).list());
}
const results = await Promise.all(promises);
const records = results.map((result) => result.records).flat();

After

const { records } = await Facet.query({ partition }).list();