To reproduce the error I'm using CloudCount and:

1. Create new document.
2. Copy new document to another computer
3. Check the "Automerge Repo" checkbox on one computer and increment count
4. At this point computer one and the repo's web socket peer state should match
5. Now on computer 2 open the document copy and check the "Automerge Repo" checkbox

At this point I would expect that second document's state to get synced with the web socket peer, but this isn't happening. To see the latest state from the peer I need to first modify the document... only then does syncing start to work.

After having updated MeetingNotes (automerge/MeetingNotes#42), the sync is clearly functioning, but its no longer happening in realtime.

When I activate websocket channel from two devices to an upstream automerge-repo server, they both appear to sync, but the content isn't replicated realtime.

When I active local peer2peer, the changes replicate faster, but on live updating of text views in MeetingNotes, there are no changes flowing back into the UI - so typing on one doesn't appear on the other.

My tests to date have been iOS <-> Mac:

• just websocket on each
• just peer-to-peer on each
• both websocket and peer-to-peer on each

The consistency of behavior makes me think there's at least some issue in the interface from automerge-repo-swift and what's happening in the docs to replicate changes, and I suspect it's on the "incoming change from remote peer" path.

If I try to connect the WebSocketProvider after it has been disconnected it always fails because peered state is set to true. Should disconnect() be resetting that state to false?

I'm periodically seeing tons of WEBSOCKET: Error reading websocket errors that grind my app to a halt. I've found one way to recreate, which is to connect the web socket provider, and then turn off my wifi-connection, but I also think the error loop can happen in other less reproducible cases.

I don't know the correct fix, but problem is happening in while true loop in ongoingReceiveWebSocketMessages. Maybe the reconnectAttempts counter needs to factor in with that error too?

I just cobbled up a bunch of WebSocket related reconnect-on-error logic, including a backoff mechanism. However, there's not really anything easily set up to "fake" a websocket to allow me to test this outside of some sort of broader integration/live testing scenario.

Go through and see if the logic can be tweaked/updated to allow injecting a fake thing that creates a WebSocketTask equivalent (or something that smells and acts like one), and use it to add tests to verify that the various permutations all work as expected.

• failed on receive()
• failed on send()
• corrupted messages
• disconnect() terminating things correctly

Boolean exposed in the configuration, but it's not yet wired up

When true, and the NWBrowser returns some peers, the system should ideally attempt to connect to those peers to establish an automatically updating/syncing connection while it's actively "listening". (Listening, and browsing, are tied together - and currently under explicit control by the hosting app)

In particular this assert fails because handle.state == .deleted:

assert(handle.state == .ready || handle.state == .unavailable || handle.state == .requesting)

To reproduce the error I'm using CloudCount and:

1. Create new document.
2. Copy new document to another computer
3. Check the "Automerge Repo" checkbox on both computers and increment count to get them syncing
4. One one of the computers uncheck the documents "Automerge Repo" checkbox
5. Then on the other computer (that's still in automerge repo) increment the change count

You will see the error on the computer where the document has been removed from automerge repository.

currently await repo.documentIds() can return partially resolved Ids, somewhat intentionally - but this can lead to flaky test results when requesting documentIds immediately after adding (some may still be in the process of storing) - so that API may need to be rethought, or the implementation resolved so that it's fully deterministic.

OSLog is a great tool, but it's supremely annoying (to me) that there isn't an API-provided way to dynamically choose logging levels. As I've been debugging things in this library, I've added logging everywhere - and it's massively overwhelming when you don't want or need it.

I came up with a ghetto version of log level that I could initialize with an instance, and this library feels like it also needs it. The alternative pattern is to only log at various levels with environment variables are set (akin to CFFoundationNetworking=3 to get the good details from the Network Framework)

Another approach is to potentially borrow from swift-log (the server-side logging framework) for both this, and Automerge-swift, in order to provide logging when asked - which does provide programmatically configurable log levels.

extract the integration tests into a separate project, within a subdirectory of this project, and run them from there

• switch from locally run ./scripts/interop.sh to running a test client in Docker
• update GH actions CI

refactor the initial websocket async code to enable automatic reconnection on network loss

• include a growing backoff on repeated failures, jittered to some level

I'm not 600% this is a fatal flaw, but it's an unintended discrepancy in the network providers.

For the WebSocket, only the "handshake" phase has an explicit timeout, and the ongoing process of receiving messages is not raced against a timeout. The idea being there may be extended periods of time (seconds, minutes, etc) where the connection is established, but nothing needs to be synced on the connection. With a timeout, the "waiting for a next message" on the connection would throw an error, and in the ongoing loop that would terminate the connection - potentially to see it attempt to reconnect if that was configured.

In the Peer2Peer provider, both the peering handshake process (both accepting a new connection and initiating a new connectIon) should use a timeout - we expect a timely handshake to establish the active connection, but there-after the ongoing loop - in either case - should require a timeout.

basic syncing is working with everything here, but it's not yet coordinated between the network providers

• add something that watches the Documents for updates (it's already ObservableObject, so that can likely be used as a trigger) in ordering to coordinate updates across the network
• sort this out with the concept of having a weak var today that can drop the connections - having something watching inside repo would make this a stronger connection in order to get the relevant updates, so we might need to be explicit abound handling memory scenarios where we'd otherwise normally purge.

Follow up from #83 and its fixes in #86:

• add a combine publisher that provides a stream of state updates of what's happening inside the WebSocket network provider so that an app using it has some information about WTF is actually happening.

• Rough collection of state values: [Connected|Ready|Waiting for Reconnect|Terminated]

• push this data from the WebSocket initial setup as well as a background task that loops to receive messages and reconnect on error (if so configured)

This reproduces the problem every time for me:

func testFind() async throws {
    let repo = Repo(sharePolicy: SharePolicy.agreeable)
    let websocket = WebSocketProvider(.init(reconnectOnError: false, loggingAt: .tracing))
    await repo.addNetworkAdapter(adapter: websocket)
    try await websocket.connect(to: URL(string: "wss://sync.automerge.org/")!)
    try await repo.find(id: .init())
}

The assert failure is in WebSocketProvider.connect at assert(peeredCopy == false).

Create an integration test to verify that the connection stays active when idle for a considerable amount of time - say 15 seconds or such - and that syncing happens immediately after that without the connection dropping and re-opening.

(This may require putting in a combine pipeline to notify - even for the test harness - of disconnects, reconnects, and current state for the WebSocket provider)

After #88 (depends on #88), add a configuration option, and associated logic in errorOnRetry, to WebSocket network provider to allow setting a maximum number of retries before "giving up" and terminating the connection.

Dependent on #88 because without it in place, if a network provider "gave up", the application has no way of knowing about it and being able to react and/or provide appropriate UI for the failed state.

refresh documentation

• curate: verify organization of all public API
• verify abstracts on all methods
• publish to GH pages

I've found two unexpected (to me) behaviors with repo.find().

First in this scenario repo.find() never finishes and never errors:

let repo = Repo(sharePolicy: SharePolicy.agreeable)
let websocket = WebSocketProvider(.init(reconnectOnError: false, loggingAt: .tracing))
await repo.addNetworkAdapter(adapter: websocket)
let handle = try await repo.find(id: DocumentId()) // never completes, never errors
print(handle)

And in this scenario (where I start web socket before repo.find()) I always successfully get a found handle with an associated document. This is unexpected because find seems to be acting more like create? I would expect some sort of "not found" error here, since the DocumentId should be unknown to the repo.

let repo = Repo(sharePolicy: SharePolicy.agreeable)
let websocket = WebSocketProvider(.init(reconnectOnError: false, loggingAt: .tracing))
await repo.addNetworkAdapter(adapter: websocket)
let handle = try await repo.find(id: DocumentId()) // always returns with a new document
print(handle)

I think maybe first is a bug. Not sure if second is expected or not.

The observing changes setup includes some race conditions that are exhibiting as flaky tests. The code isn't racing, but the test is.

ObservingChangesTest.testCreateAndObserveChange() flakes if you run it ~1000 times in quick succession, and we're seeing it intermittently on PRs with GitHub actions.

In the test, the fulfillment verifies that a sync is initiated, but doesn't await the repository "getting the sync" - there's currently no exposure of what we want "wait" on to get that, so we need to add something in there and await on both verifying the sync is initiated AND the sync has been received and processed on the receiving repository.

add GithubActions CI

• iOS build
• macOS build
• test

Right now this gives me a DuplicateID exception:

repo.create(id, doc)
repo.delete(id)
repo.create(id, doc) // DuplicateID exception

This is because after delete repo.handes still contains an internal document handle. I'm not sure if this is by design or not.

I'm running into this problem when I try to build start sharing/stop sharing commands for my document. The general idea is that when I'm sharing my document it should be in the repo, when I'm not it should not be in the repo. I understand that peers might still have a copy even when not sharing.

Add a built-in storage provider that is initialized with a fileURL, to accomodate the following use cases:

• a fileURL provided by NSDocument, UIDocument, or SwiftUI document-based APIs that points to a location on disk (for example, .fileImporter that returns a fileURL that then needs to be accessed as a security scoped URL - .startAccessingSecurityScopedResource())
• (This might be same as above, but a URL bookmark that was shared - https://developer.apple.com/documentation/security/app_sandbox/accessing_files_from_the_macos_app_sandbox)
• a constant or indicator that the storage provider should use the App's sandboxed Documents location
• an App Group container URL

• set up integration test that uses peer to peer AND websocket connections to assemble a triangle, and verify all clients get updates appropriately

depends on #48 to be resolved in order to work correctly

• convert legacy Peer2Peer networking code into a NetworkProvider plugin
• update protocol mechanisms to use CBOR encoded SYNCv1 protocol messages
• add NWBrowser and NWListener as a component of the provider to support browsing for peers and sharing that information, expose API for that detail from the network provider itself
• enable a configurable listener mode, client mode, or both

While digging on a flaky test (#104), I resolved that issue only to find that in some cases a DocumentId() generated and converted to a String (DocumentId().id) doesn't 100% of the time convert back to a valid DocumentId().

The code underneath is using BS58 encoding (Base58.base58CheckEncode(data.bytes) for data generated by UUID(). The reverse uses Base58.base58CheckDecode(stringValue) - so you'd think that would be 100%, but apparently not.

An iteration test going through 1000 of these, converting to string and trying to get back, shows a loss of 1 or 2 in 1000 (0.1 to 0.2% failure). Need to figure out why, and in the mean time I have some definite examples:

• 1YNk8bmh2xxuNqeAjsuPy4JjnwF
• 1Tf5GWML5KA18TJqUN9qKarxbhm
• 1ZEbwfpSJxXascTx7sstBecFLky
• 12mS52BJoU16LAzKmMdwgGW4tQQ
• 1kVFyeTJ6WjUmMT8pGCEesQDVxa

I'm reading through high level docs and seeing import/export API made me think there was some new concept I need to learn.

In my mind:

• Instead of import it is clearer just to use repo.create(data: data)
• Instead of using export it's clearer to just use docHandle.doc.save()

For example this seems to work without error:

let id = DocumentId()
let doc1 = Automerge.Document()
let doc2 = Automerge.Document()
_ = try await repo.create(doc: doc1, id: id)

I think without error it means that doc1 is lost from the repo and there's not really any way for original caller of repo create to detect that document instance has been replaced.

Logging updates in place, and I suspect I messed up something - WebSocket connections are now failing starting to connect, then failing - and toggling the websocket off and on again isn't explicitly resetting it. So there's a miss in there, somewhere.

Logs from MeetingNotes (trace logging enabled on WebSocket):

WEBSOCKET: Activating websocket to wss://sync.automerge.org/
WEBSOCKET: SEND: JOIN[version: 1, sender: 0711B157-AAE4-4A40-8AF5-A97375338435, metadata: [storageId: nil, ephemeral: true]]
WEBSOCKET: Attempt to send a message without a connection or defined remote peer
P2PNET: Sending SYNC[documentId: 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX, sender: 0711B157-AAE4-4A40-8AF5-A97375338435, target: storage-server-sync-automerge-org, data: 334 bytes] to peer storage-server-sync-automerge-org
P2PNET: Unable to find a connection to peer storage-server-sync-automerge-org
WEBSOCKET: Peered to targetId: storage-server-sync-automerge-org PEER[version: 1, sender: storage-server-sync-automerge-org, target: 0711B157-AAE4-4A40-8AF5-A97375338435, metadata: [storageId: 3760df37-a4c6-4f66-9ecd-732039a9385d, ephemeral: false]]
WEBSOCKET: Error reading websocket: The operation couldn’t be completed. Socket is not connected

this may be an issue with the networking subsystem as well - the error P2PNET: Unable to find a connection to peer storage-server-sync-automerge-org should have found a connection. This is notably happening on iOS, with the simulator, but not on macOS, which seemed to hold a connection without issue and clearly sent sync messages upstream as text content was changed.

currently hardcoded to 300, with a 1 second delay between checks... and makes an infinite number of reconnects on error (WebSocket)

leverage https://developer.apple.com/documentation/network/nwpathmonitor

Following up on
automerge/automerge-repo#343

I’ve learned that Unavailable isn’t a terminal state. And servers that are requested documents should track those requests so they can play them back out (during a session) if that data later becomes available, and invoke a sync to send the data that it’s now received.