Image paste support for Git repos about logseq HOT 6 CLOSED

Totally agree with this! Personally, I'm not familiar with how git works internally, but I'm happy to work with everyone to support either git-lfs or other self-host files solutions (webdav, or chrome native files API).

@tiensonqin Sounds good. It would be awesome if git-lfs support could be implemented in the future! Personally I would greatly prefer it over logseq-hosted files. The data ownership is what brings me to logseq.

I don't have experience with it, but I had a quick look at the api, and the spec for the placeholder files. I also made some quick tests in my own repo. I think it should be possible to use with your architecture without too much difficulty. (Even if it is not directly supported by isomorphic git. In fact, you probably don't want to use a native implementation since you would only want to download the attachments on demand.)

I've been thinking about how it could be implemented in logseq, and have summarized things here in case it can be useful to you in the future.

Add support to the user's repository
Add a .gitattributes file with filters like the following. (This is not necessary for logseq itself, and not for the web interface in github, but it is necessary for the files from git-lfs to show up if a user clones the repo using the native git client (while having git-lfs installed).)

*.png filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.jpeg filter=lfs diff=lfs merge=lfs -text

Image/attachment upload
When the user pastes a file, create a plaintext placeholder file in the repo according to the spec, and upload the full file to the git-lfs server using the api. (This requires two HTTP requests, but the API is relatively simple.)

Image/attachment display
When displaying a page with an embed that points to a local file, load the file from the lightningfs (like you already do with local image embeds). Check if the file is actually a placeholder (using the technique described in the spec).

• If it is a placeholder file, download the actual file contents using the api. (This requires two HTTP requests.) Then serve the resulting download using a blob link, like you already do.
• If it is determined to not be a placeholder file, just serve it directly in a blob like you already do.

Other things to think about and potential issues

• I am not sure how you do authentication towards Github in logseq, and how it relates to authentication against Github's git-lfs server. If it does not work the same way, it may be more complicated for the user to set up.
• The repository may already have a .gitattributes files with some lfs filters if the user has already set it up for git-lfs. Should be careful about what happens if the user has already added lfs-filters to other file types.
• I've not double checked, but I don't expect the necessary CORS headers to be present from the git-lfs server. You would probably need to use a cors-proxy (like you already do for the github repo itself).
• Loading each image requires two HTTP requests, and I'm not sure if the requests will be cached by the browser or not. It may therefore be a little bit slower than just passing an URL directly to an img-tag.
• The API also has endpoints for the locking of files. This confused me at first, but it seems like this is just meant to allow users to manually "lock" binary files (such as microsoft word files) to prevent others from working on the same files at the same time. In other words Logseq is not going to need to think about this locking.

Edit:
An even cleaner solution than using git-lfs would be to use partial clone (combined with some form of sparse checkout). This way, the images and other embedded files could be kept in the github repository itself, but only downloaded to the browser when they are going to be displayed on a page.

This is currently not supported by isomorphic-git, but it's being discussed at isomorphic-git/isomorphic-git#1123 .

from logseq.

I like the proposed solution here since it means everything is integrated with git.
The no-hassle solution that I like is to use is a google drive folder that I add audio, video, pdfs, photos etc to, and then copy the following into my logseq page:

<iframe src="https://drive.google.com/file/d/<enter-google-generated-random-file-id>/preview" width="640" height="480"></iframe>

All I have to do is upload the file, which is easy if I'm on mobile, or my desktop - with Gdrive desktop app, then once uploaded Get link to file, replace "https://drive.google.com/file/d/<enter-google-generated-random-file-id>/preview" with that URL and replace the view?usp=sharing with preview.
That's it and it works for many media types.
I have added two explicit examples below.

Granted there are a few steps to drag-and-drop, but I like this for several reasons:

1. My larger files are all hosted in one central location - you would get this with git-lfs too, but this needs to be set-up.
2. With the iframe class I can zoom in, scroll, or open in a new tab.
3. I don't have to use logseq storage, which I felt bad about anyway since it is a free community project. If git-lfs was set-up, then we would not have to use logseq storage either, as discussed above.
4. I can clone my repo on to another system, or mobile device, and provided I have internet connectivity, everything is accessible. Or you can install save things locally with google drive app. I imagine the same works with dropbox, one drive, S3 buckets and any other cloud-hosted file storage systems.

Note: I have tried audio files up-to 55MB and it works fine. I don't know if there is a cut-off on file size.

Working Example

Copied URL was:

https://drive.google.com/file/d/16nNW-2Obc70sXqgv7DGADdZaM3JNzUxp/view?usp=sharing

What I input on a logseq page:

<iframe src="https://drive.google.com/file/d/16nNW-2Obc70sXqgv7DGADdZaM3JNzUxp/preview" width="640" height="480"></iframe>

Other Examples

You can also embed Google sheets and other files using these templates: https://gist.github.com/tzmartin/1cf85dc3d975f94cfddc04bc0dd399be

I.e. for Google sheets, in your logseq page you could insert the following test sheet:

<iframe src="https://docs.google.com/spreadsheets/d/13R8O15c_sZKZT2QRHom1z2SDA3E1O5chUvROnqHCkwE/pubhtml?widget=true&amp;headers=true" style="width:100%;height:100%;"></iframe>

from logseq.

@bayesianbrad Pretty neat, thanks!

@tiensonqin Could this be easily implemented as an embed command same as embed youtube in logseq?

What I input on a logseq page:

<iframe src="https://drive.google.com/file/d/16nNW-2Obc70sXqgv7DGADdZaM3JNzUxp/preview" width="640" height="480"></iframe>

from logseq.

@tiensonqin I understand from your previous comments on discord that the current plan for image paste support is through uploading to logeq's servers?

The other local-first solution would be local storage in the github repo, which is already partially supported (by adding the picture to the repo manually). Is the reason for planning to use remote storage that you don't want the repos to grow too large?

If so, perhaps another option is to use something like git-lfs to store the images remotely, but on github's (or gitlab's) servers instead of logseq's servers. I am not fully familiar with how git-lfs works under the hood and how easy it would be to implement the protocol in JavaScript, but if it can be implemented I think it would be an elegant solution and more in line with the local-first spirit of logseq.

from logseq.

@hakon-j-d-johnsen

I understand from your previous comments on discord that the current plan for image paste support is through uploading to logeq's servers?

Yes, it's the current plan.

The other local-first solution would be local storage in the github repo, which is already partially supported (by adding the picture to the repo manually). Is the reason for planning to use remote storage that you don't want the repos to grow too large?

Exactly, because there'll be other files like pdfs and audios.

I am not fully familiar with how git-lfs works under the hood and how easy it would be to implement the protocol in JavaScript, but if it can be implemented I think it would be an elegant solution and more in line with the local-first spirit of logseq.

Totally agree with this! Personally, I'm not familiar with how git works internally, but I'm happy to work with everyone to support either git-lfs or other self-host files solutions (webdav, or chrome native files API).

from logseq.

希望粘贴图片在设置里面能有一个大文件提醒（比如大于2M、10M的图片）

from logseq.