Plan for the local persistence and device sync machinery (that would be compatible with sharing and real-time collab services) about pamet HOT 1 CLOSED

• Architecture overview:
  • User and service actions will produce changes to the state on each node.
    • Changes objects hold the delta between the old and new state of a leaf in the state tree. The granularity of the data held is on the level of element/page first-level properties (content/metadata/...)
  • Changes can be several per action and at the end of a top-level action (TLA) are aggregated into change-sets and a commit is produced for each of them by the indexdb repo adapter (the only local one).
    • Commit creation for the change-set is done asynchronously, since - 1. The crypto JS API is async, and 2. It's heavy computationally. So there's a copy of the state in a worker (possibly within the indexdb repo adapter) and it's there that change-sets are converted into commits with snapshot hashes.
    • The repo adapter receives the change sets via a broadcast cannel. So all local tabs, iframes, etc., write to the same queue.
      • Unpleasant details:
        • Client side services that operate with the state need to be shared across local instances - either initiated in a shared worker, or by some locking mechanism to ensure they don't do duplicate changes
        • Race conditions on the local broadcast channel are unlikely, but need to be covered by some local clock priority (at least we can count on the instances on one host to have the same clock) or locking mechanism (I prefer the latter)
    • Commits are added to the branch (and the branch head is "moved" to the last)
  • So the commits for one host are added to the same branch. But when different devices are linked (or some type of sharing is configured) - there's a branch for each node. Inter-node sync is done via the y-js WebRTC adapter.
  • Whenever a commit is added to a branch for a remote node - the local repo adopts it.
    • Each node has a seniority index (its place in the branch table/list).
    • If there's changes from several nodes (including the local frontend) in the same sync window - the commits are merged by seniority. All nodes (that made changes) beside the most senior, adopt the commits from the latter and reduce their changes to have no conflicts. In the next sync window - the former senior node does not have changes, so there's a new senior node, and the process repeats. Since the former senior node has no conflicts - on the next iteration it adopts the junior nodes commits.
      • I.e. when there's a new commit on just one branch for a sync window - the local node adopts it regardless of seniority. If there's commits on more than one branch - the most senior node waits for others to sync to it and then merges their (at this point non-conflicting) changes.
      • Regarding the "sync window" - all VCS updates (commit additions - remote and local) are pushed to a queue to be processed in the same worker (local/indexdb repo service) by a merge-handler function. So a sync-window is the span between two runs of the merge-handler.
    • When merging a commit from a more senior node - it's changes are checked for conflicts with all local commits (that come after). Since the granularity of changes in the change-sets is 1st level key-value pairs - if the same key is altered in the senior and local changes - the local changes get reduced to omit the conflicting keys. In the process a diff is produced that syncs the local state to the latest (that's a result of the merge). That effectively happens by reverting the local commits (the ones in the sync window) and applying the remote + the reduced local ones.
  • Commit squishing - after all nodes adopt a commit - the commit history behind it can be squished into it. I.e. this commit can hold the entire state. This is benefitial, since on app init the state is restored from these commits. There could be a snapshot caching mechanism, but lets not get ahead of ourselves.
  • Node dropout: since there's a max commit buffer (memory constraint) and we assume active nodes will sync often enough - if a node (regardless of seniority) has not synced for a while - it can be "dropped out". Not necessarily deauthorized but we treat it as a junior and resync it with the latest state before restoring it's normal status. There may be an explicit approval by the user for this.

from pamet.