Comments (8)
Very nice! I'm happy it works now. 🙌
from hocuspocus.
One comment on your implementation:
async onChange(data) {
const save = async () => {
// ...
}
// you can remove this, it's basically the same as this._debounced?.clear()
if (this._debounced) {
this._debounced.clear();
}
this._debounced?.clear();
this._debounced = debounce(() => save, this._debounceTime);
this._debounced();
// debouncing is meant to take load off your database server
// calling save here directly will just make the whole debouncing useless
save();
}
from hocuspocus.
Hey Zeljko. Thank you so much for your bug report. Can you share your backend/hocuspocus code as well?
from hocuspocus.
Do you use the @hocuspocus/extension-rocksdb
as primary storage? If not, you definitely should.
If you do, do you check in your implementation of the onCreateDocument
hook if the document is already present in the primary storage?
const hocuspocus = Server.configure({
async onCreateDocument(data) {
const fieldName = 'default'
// Check if the given field already exists in the given document.
// Important: Only import a document if it doesn't exist in the primary data storage!
if (data.document.isEmpty(fieldName)) {
return
}
// import the document from somewhere else…
return new Doc()
}
}
The background is: Serializing the Y-Doc to Prosemirror JSON in the onChange
hook and deserializing it in the onCreateDocument
hook, doesn't save the collaboration history. And thus, Yjs doesn't know that those changes are the same and just merges them.
Because saving a Y-Doc with the related collaboration history can be hard, we built the @hocuspocus/extension-rocksdb
which is meant to be your primary storage and makes this incredibly easy (it's just one line to implement).
You can read more on the need for a primary storage here: https://www.hocuspocus.dev/guide/documents#using-a-primary-storage
from hocuspocus.
Hi Kris, here is the collab server code:
class CollaborationServer {
_server = null;
_debounced = null;
_debounceTime = 4000;
constructor() {
this._server = Server.configure({
port: process.env.PORT || 80,
// allows up to 5 connection attempts per ip address per minute.
// set to null or false to disable throttling
throttle: false,
// bans ip addresses for 5 minutes after reaching the throttling threshold
// banTime: 500,
onConnect: this.onConnect,
onCreateDocument: this.onCreateDocument,
onChange: this.onChange
});
}
async listen() {
return this._server.listen();
}
async onConnect(data) {
const { requestParameters } = data;
const token = requestParameters.get('token');
if (!token) {
throw new Error('Auth token not provided');
}
const user = await getUserByToken(token);
if (!user) {
throw new Error('Authentication failed');
}
// You can set contextual data to use it in other hooks
return {
user: {
id: user.id,
name: user.fullName,
},
}
}
async onCreateDocument(data) {
// The tiptap collaboration extension uses shared types of a single y-doc
// to store different fields in the same document.
// The default field in tiptap is simply called "default"
const fieldName = 'default';
// Check if the given field already exists in the given y-doc.
// Only import a document if it doesn't exist in the primary data storage
if (data.document.get(fieldName)._start) {
return;
}
const namespace = data.documentName.split('/');
if (namespace.length !== 2) {
return;
}
const [ projectId, itemId ] = namespace;
const prosemirrorDocument = await getItem(itemId);
if (!prosemirrorDocument) {
return;
}
// When using the tiptap editor we need the schema to create
// a prosemirror JSON. We must list all extensions that are used on the front-end
const schema = getSchema([...defaultExtensions(), Image.configure({inline: false}), Link, Underline]);
// Convert the prosemirror JSON to a ydoc and simply return it.
return prosemirrorJSONToYDoc(schema, prosemirrorDocument, fieldName);
}
async onChange(data) {
const save = async () => {
// Get the underlying Y-Doc
const ydoc = data.document;
// Convert the y-doc to the format your editor uses, in this
// example Prosemirror JSON for the tiptap editor
const prosemirrorDocument = yDocToProsemirrorJSON(ydoc, 'default');
const [ projectId, itemId ] = data.documentName.split('/');;
const newContent = prosemirrorDocument;
// Updates database
await updateItemContent(itemId, newContent);
};
if (this._debounced) {
this._debounced.clear();
}
this._debounced?.clear();
this._debounced = debounce(() => save, this._debounceTime);
this._debounced();
save();
}
}
export {
CollaborationServer
};
It seems that i miss-understood the purpose of the rocksDb, i thought that it can be used instead of PostgreSQL for example (which i am using). I didn't find any tutorial regarding the mix of rocksDb with other databases, that's why i didn't include it and used PostgreSQL directly (is that wrong?).
Also, i can't find any example how to configure UI for using IndexedDB with v2.. I found in example that it is imported, but not set anywhere (or importing is enough? i was looking https://next.tiptap.dev/examples/collaborative-editing)
from hocuspocus.
I didn't find any tutorial regarding the mix of rocksDb with other databases, that's why i didn't include it and used PostgreSQL directly (is that wrong?).
It's not wrong per se, but it will lead to issues like the one you described above. Your setup looks fine, just include the rocksdb extension and it should work. :)
Theoretically, you could also store the collaboration history in your PostgreSQL database, but it's hard. Especially because there is no documentation at the moment on how to do it. We ourselves just kinda brute-forced our implementation: https://docs.yjs.dev/tutorials/persisting-the-document-to-a-central-database
Also, i can't find any example how to configure UI for using IndexedDB with v2.. I found in example that it is imported, but not set anywhere (or importing is enough? i was looking https://next.tiptap.dev/examples/collaborative-editing)
I recently removed the indexedDB from tiptaps collaboration example because we had some issues with it and hocuspocus. Long story short: We completely changed how hocuspocus worked a few weeks back and it resulted in corrupted data which was stored in the clients indexeddb and completely destroyed our collab-system because Y-js always wanted to merge the corrupted data… So, nothing to worry about for you! It's pretty simple to set up, just import it, create a new instance and pass it the Y-Doc:
import { IndexeddbPersistence } from 'y-indexeddb'
const ydoc = new Y.Doc()
const indexdb = new IndexeddbPersistence('tiptap-collaboration-example', ydoc)
from hocuspocus.
Make sure to use the latest version of @hocuspocus/extension-rocksdb
(>= 1.0.0-alpha.42
) as we fixed some issues with the underlying rocksdb adapter in this release. If you experience errors after updating, delete your database folder and restart the server.
from hocuspocus.
Hi Kris,
It works very good now! Thank you for suggestions!
from hocuspocus.
Related Issues (20)
- How can a provider know if the document existed or not HOT 1
- onConnect payload includes `context` but type signature does not reveal it HOT 1
- onRequest hook should allow to write custom status and response HOT 1
- Documents loaded via direct connection not unloaded on disconnect
- Graceful shutdown HOT 1
- Connection check in multiplexing HOT 7
- Connection getting closed with "Unauthorized" in Firefox and Safari after about 30 seconds HOT 5
- Error playground in gh codespaces
- Server has websocket.binarytype set to arraybuffer, not nodebuffer HOT 4
- Support multiple server instances communicating with each other in HA setup HOT 1
- onMessage is inefficiently called on every Doc
- Content inserted one time per connection HOT 1
- handleDocumentUpdate does not check if connection is valid correctly HOT 3
- What will happen if we don't store whole ydoc HOT 2
- Is it possible for provider to fetch document by default HOT 1
- Documents are not stored before server shut down when `unloadImmediately` is set to false HOT 1
- Too many keystrokes break Websocket connection HOT 1
- appName in Webhook comes empty HOT 1
- Feature Request - Server logs
- Maybe need enable perMessageDeflate
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hocuspocus.