onCreateDocument does not have the client document's data about hocuspocus HOT 16 CLOSED

This is more important now that we have seen a few YJS documents become corrupted causing changes to be applied oddly.

If we could create the document with the first connection and dispose it when all connections disconnect, then it would reduce the document lifetime and avoid corruption.

from hocuspocus.

It would be nice to have a hook for the "If the user started typing already, don’t care about getting content from an external data source".

That way if RocksDB data gets corrupted for some reason, reinitialization can be done safely.

from hocuspocus.

Sorry, for the delay here.

Currently, I’m on holiday. I’ll tackle this in August (and will generally put more time into hocuspocus starting September).

from hocuspocus.

I’m not really sure whether that’s a bug or a feature. We assume the following flow:

1. Check if there’s a Y.js document stored
2. If there’s no Y.js document, create one, optionally based on an external data source (API, database, whatever …)
3. While the server is busy looking and creating the document, the user can start typing
4. Once the document is there, merge everything, nothing is lost

Does that make sense?

In your expected behaviour you’d say: If the user started typing already, don’t care about getting content from an external data source. Right?

There’s probably a need for the second use case, but I think we’d need two different hooks to handle both scenarios.

from hocuspocus.

To avoid needing to store the documents, I was planning to use onCreateDocument to initialize the document from our API. After initializing, then I was going to set a field initialized.

If that field is already set, it will not initialize the content from our servers. That way if the server restarts due to some failure, etc, it will not reinitialize the document (causing duplicate text) if the client already received the document.

I am doing this now with a setTimeout as a workaround and the changes already exist.

setTimeout(() => context.controller.initialize(document), 100);

from hocuspocus.

That might work for your use case, maybe we should talk this through. But generally speaking: Not storing the history of the Y.js document (for example in RocksDB), is asking for trouble. :)

The purpose of the onCreateDocument hook is to initially migrate data from an external data source. After that initial migration, I’d recommend to keep the Y.js history. Only with that history you’ve got all the benefits that Y.js provides, for example multiple people working offline, or with a bumpy connection.

I can understand that you’d like to avoid storing something in RocksDB (something new, another piece of infrastructure …), and I think that applies to a lot of people. I think we’d need to provide a persistent extension for a more common database. Currently, I’m considering to write a PostgreSQL extension. What’s the database and storage tech stack you’re used to?

from hocuspocus.

Our main use case for Y.js is to allow multi-user collaboration while they are online. So we're OK locking the document while they are offline in the short term to simplify our implementation.

In the future (but no an immediate priority) we'd like to store version history and allow users to rollback to previous states. If that was possible with RocksDB we'd be happy to add it to our stack.

Right now we are using PostgreSQL.

from hocuspocus.

We use Webhook extension to retrieve document from database. And we encounter an issue that our document content will duplicate when our editor lost the connection and reconnected to server. I think our situation may related to this issue because it pass the isEmpty check in Webhook extension

from hocuspocus.

@hanspagel to further comment from @DanSnow, do you have any suggested workarounds to this issue?

from hocuspocus.

Oh, that’s a bug, yes! I’ll try to fix this. Still not sure whether I should change that hook, or introduce a new one though.

from hocuspocus.

hey @hanspagel, is there a rough timeframe/plan for fixing this bug? Apologies for following up on this, but some guidance would really help our implementation of this awesome library!

from hocuspocus.

I'm confident this should not be considered a bug, with yjs you have to adjust your way of thinking towards distributed systems as it will make your application more reliable in the long run. If possible it's best to think of the server as just another node in the distributed doc, it's just one that happens to also host the application and maybe trigger some notifications etc 😄

onCreateDocument is a hook to get the last persisted state on the server (if any), it is called after authentication and before the first client is fully connected, as such it doesn't make sense to have the client's state. In fact, if you're using another local provider such as y-indexeddb there is no guarantee that the clients "local state" will be loaded before the server's is!

from hocuspocus.

@tommoor yep agreed re "the server as just another node in the distributed doc".

However, the documentation for HocusPocus says that we need to get the persisted state in the onCreateDocument hook, so either this bug should be resolved or the documentation be updated 😅

from hocuspocus.

Sorry, I'm not following how the docs are invalid sorry, you should still load the persisted state in the onCreateDocument hook

from hocuspocus.

@tommoor iirc the docs state we should load the persisted article in onCreateDocument hook, so either the docs are invalid or there is a bug in code

from hocuspocus.

I can’t find a way to make sure the client data is present in the backend at that time, due to the nature of Y.js.

I think the example was pretty misleading, so I rewrote that doc page. Hope that helps a little bit at least.

from hocuspocus.