Giter VIP home page Giter VIP logo

Comments (80)

robwheatley avatar robwheatley commented on July 22, 2024 1

@brianpetro here is the 1st screenshot - taken after start-up with the debugging on. The 2nd screenshot is from after I clicked the 'create embeddings' button up to the point it finished

Shot 1
Screenshot 2024-03-05 at 14 50 39

Shot 2
Screenshot 2024-03-05 at 14 58 12

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024 1

Thinking out loud here:

Something else that was recently changed was replacing a file hash (b/c incompatible with mobile) with checking both the files size and last change time.

In theory, this should not cause a noticeable difference because even if the time is modified by some other process, the file size should stay the same.

But, in practice, maybe the file size is also being altered even without changing the note.

I'm going to need to come up with some sort of test for this.

🌴

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@bbecausereasonss are you talking about the "save" by the API key? There isn't currently a method for manually saving. However, checking the developer console logs should let us know whether the file is being saved. It should say something like "Saved in XXXXms".

from obsidian-smart-connections.

bbecausereasonss avatar bbecausereasonss commented on July 22, 2024

No I mean the save .smart-connections data

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

That button simply saves the setting. It's a manual save for the setting because if you change the folder, it needs to rename it, which you don't want to trigger automatically.

from obsidian-smart-connections.

DantesHub avatar DantesHub commented on July 22, 2024

CleanShot 2024-03-04 at 20 28 27
Hey @brianpetro I'm having the same problem, this prompts keeps appearing everytime i reopen obsidian.

from obsidian-smart-connections.

DantesHub avatar DantesHub commented on July 22, 2024

CleanShot 2024-03-04 at 20 30 03

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

I have the same problem. My vault is saved to iCloud rather than dropbox and I assume it has something to do with the file dates. I'm wondering if iCloud/DropBox messes with the dates on files so Smart Connections things that the embeddings are out of date. I came here to see if that was the case. Not found anything yet, but perhaps this will give people a clue?

from obsidian-smart-connections.

sh4d0wl3ss avatar sh4d0wl3ss commented on July 22, 2024

Same issue and I am using Obisidan Sync but also generate daily journal entries and have about 8k notes in the vault. I'm spending much more time with degraded Obsidian performance while it embeds smart notes on the same folder every day than I actually use the smart connections features. Juice isn't worth the squeeze. Didn't do this before the latest Obsidian update.

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley @DantesHub @sh4d0wl3ss

Keep an eye on the dev console during the embedding process. If there's an error with saving, screenshot and I can fix that.

If everything saves correctly, then something else is clearing the embeddings. Check the dev console immediately after startup to check for loading issues.

Otherwise, it might be something that's specific to your setup. So also include any third-party syncing that you're using, like how @robwheatley mentioned he is using iCloud.

Thanks for your help in solving this issue,
🌴

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

Also, which operating system you're using will help narrow down the issue.

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

Hi @brianpetro I'm new to Obsidian and didn't realise that there was a dev console. I've just taken a look and have spotted a few errors being thrown out by smart connections. Not sure if this helps you, but I can provide more info on request...

I'm on oSX 14.3.1 and Obsidian 1.5.8 and smart connections 1.0.128. Also, as mentioned, I have my vault in iCloud.

As a test, I moved my vault out of iCloud and onto my regular drive. When I did that, the dev log did look a little different - I more entries for "Embedded X inputs..." but I still got the final undefined 'last_history' at the end of the log like below. Also, on restart of Obsidian, all my notes required embedding again - so it doesn't look like an iCloud specific problem.

Here is the log when running on iCloud...

app.js:1 Obsidian Developer Console
plugin:obsidian-tasks-plugin:149 [2024-03-05-13:16:53.170][info][tasks] loading plugin "Tasks" v6.0.0 
plugin:dataview:20020 Dataview: version 0.5.64 (requires obsidian 0.13.11)
plugin:table-editor-obsidian:28854 loading markdown-table-editor plugin
plugin:obsidian-icon-folder:5427 loading iconize
plugin:quickadd:19868 Loading QuickAdd
plugin:quickadd:17812 QuickAdd: (LOG) No migrations to run.
plugin:highlightr-plugin:3508 Highlightr v1.2.2 loaded
plugin:smart-connections:6177 Loading Smart Connections v2...
plugin:dataview:12759 Dataview: all 827 files have been indexed in 0.133s (827 cached, 0 skipped).
plugin:smart-connections:2596 WebSocket connection to 'ws://localhost:37042/' failed: 
eval @ plugin:smart-connections:2596
plugin:smart-connections:2618 Websocket error Event
on_error @ plugin:smart-connections:2618
plugin:smart-connections:2614 Failed to connect to WebSocket: Event
connect_to_websocket @ plugin:smart-connections:2614
plugin:smart-connections:2673 DataviewSocket initialized
plugin:smart-connections:2709 Disconnected from websocket 
plugin:obsidian-icon-folder:594 loaded icon pack boxicons (1634)
plugin:obsidian-icon-folder:594 loaded icon pack coolicons (442)
plugin:obsidian-icon-folder:594 loaded icon pack font-awesome-regular (163)
plugin:obsidian-icon-folder:594 loaded icon pack remix-icons (2736)
plugin:smart-connections:2065 Web
plugin:smart-connections:2065 Web
SmartEmbedTransformersWebConnector.js:20 Loading Smart Local Model
SmartEmbedTransformersWebConnector.js:20 Loading Smart Local Model
transformers@latest:34 worker sent an error! blob:app://obsidian.md/b86e45ac-6ea6-4899-8684-cf5bbe6b9f09:1: Uncaught TypeError: Cannot read properties of null (reading 'on')
Fc.e.onerror @ transformers@latest:34
transformers@latest:34 Uncaught ErrorEvent
b86e45ac-6ea6-4899-8684-cf5bbe6b9f09:1 Uncaught TypeError: Cannot read properties of null (reading 'on')
    at b86e45ac-6ea6-4899-8684-cf5bbe6b9f09:1:180
transformers@latest:34 worker sent an error! blob:app://obsidian.md/ac1fcd1e-0943-4e61-b832-f8cc140ca13d:1: Uncaught TypeError: Cannot read properties of null (reading 'on')
Fc.e.onerror @ transformers@latest:34
transformers@latest:34 Uncaught ErrorEvent
ac1fcd1e-0943-4e61-b832-f8cc140ca13d:1 Uncaught TypeError: Cannot read properties of null (reading 'on')
    at ac1fcd1e-0943-4e61-b832-f8cc140ca13d:1:180
transformers@latest:34 worker sent an error! blob:app://obsidian.md/680df758-71a9-481a-8120-01d07ee32d12:1: Uncaught TypeError: Cannot read properties of null (reading 'on')
Fc.e.onerror @ transformers@latest:34
transformers@latest:34 Uncaught ErrorEvent
transformers@latest:34 worker sent an error! blob:app://obsidian.md/2aa177f7-9034-46cf-8567-ed2b0c93df37:1: Uncaught TypeError: Cannot read properties of null (reading 'on')
Fc.e.onerror @ transformers@latest:34
680df758-71a9-481a-8120-01d07ee32d12:1 Uncaught TypeError: Cannot read properties of null (reading 'on')
    at 680df758-71a9-481a-8120-01d07ee32d12:1:180
transformers@latest:34 Uncaught ErrorEvent
2aa177f7-9034-46cf-8567-ed2b0c93df37:1 Uncaught TypeError: Cannot read properties of null (reading 'on')
    at 2aa177f7-9034-46cf-8567-ed2b0c93df37:1:180
transformers@latest:34 worker sent an error! blob:app://obsidian.md/cbb451c0-eb9d-4c33-afb2-99922b949493:1: Uncaught TypeError: Cannot read properties of null (reading 'on')
Fc.e.onerror @ transformers@latest:34
transformers@latest:34 Uncaught ErrorEvent
cbb451c0-eb9d-4c33-afb2-99922b949493:1 Uncaught TypeError: Cannot read properties of null (reading 'on')
    at cbb451c0-eb9d-4c33-afb2-99922b949493:1:180
transformers@latest:34 worker sent an error! blob:app://obsidian.md/f42cf518-a17f-48bf-8a87-0fcce6ee7f16:1: Uncaught TypeError: Cannot read properties of null (reading 'on')
Fc.e.onerror @ transformers@latest:34
transformers@latest:34 Uncaught ErrorEvent
f42cf518-a17f-48bf-8a87-0fcce6ee7f16:1 Uncaught TypeError: Cannot read properties of null (reading 'on')
    at f42cf518-a17f-48bf-8a87-0fcce6ee7f16:1:180
SmartEmbedTransformersWebConnector.js:28 Object
plugin:smart-connections:1690 Model Loaded: Xenova/jina-embeddings-v2-small-en
plugin:smart-connections:1702 SmartEmbedTransformersWebAdapter Connected
plugin:smart-connections:103 Loading: .smart-connections/smart_notes-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:1690 Model Loaded: Xenova/jina-embeddings-v2-small-en
plugin:smart-connections:1702 SmartEmbedTransformersWebAdapter Connected
plugin:smart-connections:103 Loading: .smart-connections/smart_blocks-jina-embeddings-v2-small-en-2048.ajson
SmartEmbedTransformersWebConnector.js:28 Objectembed_input: "test"tokens: 3vec: (512) [-0.04618043, -0.06881742, 0.04266885, 0.05441353, 0.02042668, -0.02698513, 0.04688265, 0.04704173, -0.00598965, -0.00829872, 0.02351636, -0.00252418, -0.08863518, -0.12731825, 0.10426091, -0.07812816, 0.00123997, -0.01867516, -0.03606192, -0.02378379, -0.02312022, -0.02576988, -0.0064623, -0.04856632, 0.04698525, -0.05169244, -0.05126242, -0.00170323, 0.06596053, 0.04149962, -0.07831065, 0.02595678, -0.02589971, -0.02380194, -0.01952333, -0.01083052, 0.05182994, 0.03434894, -0.00794646, -0.0485135, -0.04945732, 0.008254, 0.05211577, 0.01925983, -0.03337441, 0.04778156, -0.05378807, 0.02331959, 0.04884573, -0.04831477, -0.04477754, 0.02440652, -0.06886175, 0.04624782, 0.0006633, -0.00101584, -0.06547856, 0.02407879, -0.03827589, 0.04307167, -0.0093802, 0.01067624, -0.03204694, 0.07326417, -0.01929409, -0.04841447, -0.05658561, -0.02511995, -0.0901373, 0.00482804, -0.05070149, -0.01075389, 0.08151168, -0.03977614, 0.01376666, 0.03035205, -0.0226281, 0.06229437, 0.02023081, -0.02384315, -0.05419815, -0.03349979, -0.02139989, 0.03281472, 0.02099423, -0.07024483, 0.02373451, 0.07234769, 0.07402453, -0.01802289, 0.01551048, 0.05586207, -0.01492319, -0.06774362, -0.00859283, -0.00702043, 0.05499087, 0.02260393, -0.0021433, 0.03894763, …][[Prototype]]: Object
plugin:smart-connections:111 Error loading: .smart-connections/smart_notes-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:112 SyntaxError: Bad control character in string literal in JSON at position 3043690 (line 1 column 3043691)
    at JSON.parse (<anonymous>)
    at ObsAJSON2.load (plugin:smart-connections:105:31)
    at async SmartNotes.load (plugin:smart-connections:2054:9)
    at async eval (plugin:smart-connections:2767:92)
    at async Promise.all (index 0)
    at async ScBrain2.init (plugin:smart-connections:2767:9)
    at async ScBrain2.reload (plugin:smart-connections:2750:9)
    at async SmartView2.load_brain (plugin:smart-connections:4293:9)
    at async SmartView2.initialize (plugin:smart-connections:4283:9)
plugin:smart-connections:111 Error loading: .smart-connections/smart_blocks-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:112 Error: ENOENT: no such file or directory, open '/Users/robwheatley/Library/Mobile Documents/iCloud~md~obsidian/Documents/MainVault/.smart-connections/smart_blocks-jina-embeddings-v2-small-en-2048.ajson'
plugin:smart-connections:145 Saving: smart_notes-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:168 Saved smart_notes-jina-embeddings-v2-small-en-2048.ajson in 28ms
plugin:obsidian-icon-folder:594 loaded icon pack simple-icons (2973)
plugin:smart-connections:145 Saving: smart_notes-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:145 Saving: smart_blocks-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:168 Saved smart_notes-jina-embeddings-v2-small-en-2048.ajson in 407ms
plugin:smart-connections:168 Saved smart_blocks-jina-embeddings-v2-small-en-2048.ajson in 408ms
SmartEmbedTransformersWebConnector.js:61 Embedded: 25 inputs (8679 tokens, 862 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 33 inputs (9228 tokens, 875 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 58 inputs (9989 tokens, 984 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 40 inputs (9566 tokens, 952 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 40 inputs (9908 tokens, 981 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 39 inputs (9710 tokens, 957 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 52 inputs (10177 tokens, 955 tokens/sec)
SmartEmbedTransformersNodeAdapter.js:26 Input too long. Truncating to  4482  characters.
SmartEmbedTransformersNodeAdapter.js:27 Tokens:  5115  ->  1869
SmartEmbedTransformersWebConnector.js:61 Embedded: 21 inputs (9568 tokens, 839 tokens/sec)
SmartEmbedTransformersNodeAdapter.js:26 Input too long. Truncating to  3449  characters.
SmartEmbedTransformersNodeAdapter.js:27 Tokens:  2264  ->  1833
SmartEmbedTransformersWebConnector.js:61 Embedded: 42 inputs (9311 tokens, 922 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 51 inputs (10321 tokens, 987 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 34 inputs (9360 tokens, 923 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 56 inputs (10115 tokens, 999 tokens/sec)
plugin:smart-connections:145 Saving: smart_notes-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:168 Saved smart_notes-jina-embeddings-v2-small-en-2048.ajson in 60ms
SmartEmbedTransformersWebConnector.js:61 Embedded: 46 inputs (9460 tokens, 943 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 41 inputs (10208 tokens, 977 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 35 inputs (9976 tokens, 997 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 25 inputs (8096 tokens, 802 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 14 inputs (5420 tokens, 536 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 20 inputs (5321 tokens, 507 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 17 inputs (5861 tokens, 558 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 26 inputs (5660 tokens, 559 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 22 inputs (5728 tokens, 538 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 24 inputs (5822 tokens, 544 tokens/sec)
SmartEmbedTransformersNodeAdapter.js:26 Input too long. Truncating to  7333  characters.
SmartEmbedTransformersNodeAdapter.js:27 Tokens:  2586  ->  1867
SmartEmbedTransformersWebConnector.js:61 Embedded: 14 inputs (4838 tokens, 481 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 9 inputs (5054 tokens, 425 tokens/sec)
SmartEmbedTransformersNodeAdapter.js:26 Input too long. Truncating to  5017  characters.
SmartEmbedTransformersNodeAdapter.js:27 Tokens:  5758  ->  1762
SmartEmbedTransformersWebConnector.js:61 Embedded: 16 inputs (5386 tokens, 494 tokens/sec)
SmartEmbedTransformersWebConnector.js:61 Embedded: 20 inputs (5694 tokens, 526 tokens/sec)
plugin:smart-connections:145 Saving: smart_notes-jina-embeddings-v2-small-en-2048.ajson
plugin:smart-connections:2507 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'last_history')
    at SmartBlock.init (plugin:smart-connections:2507:37)
    at SmartBlocks.create_or_update (plugin:smart-connections:750:14)
    at eval (plugin:smart-connections:2447:42)
    at Array.forEach (<anonymous>)
    at eval (plugin:smart-connections:2447:18)
    at async Promise.all (index 278)
    at async SmartBlocks.import (plugin:smart-connections:2444:9)
init @ plugin:smart-connections:2507
create_or_update @ plugin:smart-connections:750
eval @ plugin:smart-connections:2447
eval @ plugin:smart-connections:2447
await in eval (async)
ensure_embeddings @ plugin:smart-connections:2264
postMessage (async)
embed_handler @ SmartEmbedTransformersWebConnector.js:48
await in embed_handler (async)
handle_ipc @ SmartEmbedTransformersWebConnector.js:33
postMessage (async)
request_embedding @ plugin:smart-connections:1708
embed_batch @ plugin:smart-connections:1733
ensure_embeddings @ plugin:smart-connections:2121
postMessage (async)
embed_handler @ SmartEmbedTransformersWebConnector.js:48
await in embed_handler (async)
handle_ipc @ SmartEmbedTransformersWebConnector.js:33
plugin:smart-connections:168 Saved smart_notes-jina-embeddings-v2-small-en-2048.ajson in 135ms

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley if you could screenshot the console, that would be much easier for me to look through.

Also, toggle this on for better logs
Screenshot 2024-02-02 at 10 34 25 PM
🌴

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

Does anyone that this has happened to remember hitting the pause button prior to losing the embeddings?

It seems like under some conditions, like pausing/restarting, multiple embedding processes could be executed at once. This is visualized by the denominator in the progress notification changing between multiple values.

So far this has only happened once for me during development, so it'll require more testing.

There could be some other situations where this happens. I'm continuing to investigate.

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

Does anyone that this has happened to remember hitting the pause button prior to losing the embeddings?

It seems like under some conditions, like pausing/restarting, multiple embedding processes could be executed at once. This is visualized by the denominator in the progress notification changing between multiple values.

So far this has only happened once for me during development, so it'll require more testing.

There could be some other situations where this happens. I'm continuing to investigate.

I'm 99% sure I didn't pause. OK, 98%

from obsidian-smart-connections.

bbecausereasonss avatar bbecausereasonss commented on July 22, 2024

Does anyone that this has happened to remember hitting the pause button prior to losing the embeddings?

It seems like under some conditions, like pausing/restarting, multiple embedding processes could be executed at once. This is visualized by the denominator in the progress notification changing between multiple values.

So far this has only happened once for me during development, so it'll require more testing.

There could be some other situations where this happens. I'm continuing to investigate.

I hit the pause button once, but that was weeks ago and it's been happening ever since.

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley @bbecausereasonss thanks for letting me know!

🌴

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

I've been doing some messing about with a new Vault saved directly onto my hard drive. Clean install, with only the Smart Connections plugin installed. And I've been adding notes to that to see what it does....

I think what's happening is that when the embeddings.ajson file is loaded on start-up, the JSAON parser doesn't like something and reports the error below. That results in the embedding.ajson file being deleted, so you have to start the embedding again.

SyntaxError: Bad control character in string literal in JSON at position 3054377 (line 1 column 3054378)
    at JSON.parse (<anonymous>)
    at ObsAJSON2.load (plugin:smart-connections:105:31)
    at async SmartNotes.load (plugin:smart-connections:2054:9)
    at async eval (plugin:smart-connections:2767:92)
    at async Promise.all (index 0)
    at async ScBrain2.init (plugin:smart-connections:2767:9)
    at async ScBrain2.reload (plugin:smart-connections:2750:9)
    at async SmartView2.load_brain (plugin:smart-connections:4293:9)
    at async SmartView2.initialize (plugin:smart-connections:4283:9)

I don't know what the bad character is - I can't see anything obviously wrong with the file (I can do more testing later) and I don't know what put it here in the first place. I wonder if I could hack something to prevent the file from being deleted when it discovers the error to see what's going on....?

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley good catch!

That error doesn't specifically delete the file, but the embeddings fail to load and then the reprocessing overwrites the existing file.

Solving the source of the issue: I'm thinking it's this line

obsidian-smart-connections/src/smart_entities.js

Line 165 in 077bf4f

get ajson() { return `"${this.key.replace(/"/g, '\\"')}": ${JSON.stringify(this.data)}`; }

Specifically, "${this.key.replace(/"/g, '\\"')}": should be ${JSON.stringify(this.key)} to handle any control characters.

I'll get this change shipped in the next update, today if I can fit it in.

Another thing that can help situations like this: saving the file so that records (or batches of records) are separated by newlines. This way the erroneous record/batch can be thrown out while preserving the rest. This would likely have a negative impact on start-up performance, but would still probably be worth it to prevent this embedding-rewrite headache.

🌴

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

@brianpetro It's great that you are looking into this. I've just spent the last few hours seeing if I could add anymore info. I went down a bit of a rabbithole TBH!

From my clean install, I started to add notes in from my 'real' vault. I wondered if a particular note was causing the issue. After lots of messing about I thought I found something. When I added a specific note, I started to get errors. But it turned out to be nothing special about that note. If I just added 'one more note' of any sort, I would cause the issue. Basically, I got in the situation where I had 236 notes, but adding a 237th would make things fail.

I then started to look other things, because adding the 237th note doesn't 100% reproduce the problem. So I then started to add more content to notes when I just had 236. I'm not convinced that this actually got me anywhere though!!

I did run into a few odd things along the way though. For example, when I added a new note, I got the alert to say that it was being embedded, but the alert never went away, even though I could see in the console that some sort of embedding had been done. Sometimes I saw a time-out on this single file (dunno why, I was using a super simple embedding and I'm on a speedy machine). Also, on start-up, I sometimes get asked if I want to re-embed all my notes, even though I can see there is a valid embeddings file and there has been no parsing errors. Quit and restart sorts that out (on next run, I'm not asked to re-embed). I'm not sure how you are keeping track of what's been embedded or not. Maybe the embeddings file itself, and these issues were being caused by file-on-disk mismatches. No idea, and I realise these ramblings won't help!

I'm super-keen to get this plugin working though. I've only just moved to Obsidian, and although I'm putting some structure in place for new notes, the old ones I have imported are a mess, so this would be really useful!

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley thanks for sharing all that! Your rabbit hole can be my gold mine. It's not very often (considering how many people have downloaded Smart Connections) that I get such detailed feedback 😊

You definitely managed to point out some curiosities.

Separating meta data from embeddings files is something I've played around with in the past, and could be a way to thwart some of these issues.

If you ever have a note that seems to cause an issue, but you can't figure out why, please do share the note with me. If you need a private channel to do so, I can accommodate. But being able to see some of these issues myself can be invaluable to the debugging process.

There is still a lot of legacy code in v2.0, but I'm continuing to modularize the processes, enabling useful test processes, so the stability will only improve (though things tend to get worse before they get better...). More importantly, these design decisions should also allow contributing by community members long into the future.

PS- all these GitHub issues end up in my personal obsidian vault, and Smart Connections enables me to resurface them at the right times. So any notes you make on your experience, much like what you just shared, will be useful even if they aren't specifically addressed right away.

Thanks for your help in making Smart Connections better!
🌴

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley @bbecausereasonss @DantesHub @sh4d0wl3ss latest update v2.0.129 implements #496 (comment)

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

@brianpetro No joy I'm afraid. I updated to the latest version, scrubbed everything and started embedding from scratch.

My local vault created the embeddings file, but on restart wants me to re-do them all again (even though there is a valid 2Meg file there, that doesn't get overwritten). I can get more from the logs on this later.

My iCloud vault created the embeddings (took at lot longer as I have 4x more notes in this one), it said that it saved the file in the console, but the saved file was zero bytes, and obviously I get asked to re-do them on restart.

Must be something else causing the issue. I don't think I will have much time to play tonight, but will over the weekend if you don't work it out before then....

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley bummer, but thanks for letting me know.

When you get a chance, let me know if you're still seeing the same error or if it's something new.

🌴

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley @bbecausereasonss @DantesHub @sh4d0wl3ss released another update v2.0.130 to try to address this. In short, a temporary file is created on save to prevent overwriting the existing file 🌴

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

@brianpetro I tried .30 and it looked liked it worked, but saw a couple of funny things. I wasn't paying much attention as I was waiting in the car for my daughter. Just got back in the house and seen that you have released a .31 version.

Happy to report that I've hit no issues so far (local or iCloud). To flex it a bit, I've just switched to a beefier model to see if a larger file size causes any issues. It's chugging away as I type and I can see that you are saving the file after a few notes have been processed and recording the file sizes as you go. At least I can see the size increasing, so that's encouraging!

YES! IT WORKED - Nice one!!
I've quit and restarted several times and not been asked to re-embed everything again

from obsidian-smart-connections.

robwheatley avatar robwheatley commented on July 22, 2024

@brianpetro Spoke too soon. It creates the notes embeddings fine, but it's failing to create block embeddings at the moment. I'm getting this error. Just thought I should let you know.

plugin:smart-connections:2520 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'last_history')
    at SmartBlock.init (plugin:smart-connections:2520:37)
    at SmartBlocks.create_or_update (plugin:smart-connections:763:14)
    at eval (plugin:smart-connections:2460:42)
    at Array.forEach (<anonymous>)
    at eval (plugin:smart-connections:2460:18)
    at async Promise.all (index 284)
    at async SmartBlocks.import (plugin:smart-connections:2457:9)
init @ plugin:smart-connections:2520
create_or_update @ plugin:smart-connections:763
eval @ plugin:smart-connections:2460
eval @ plugin:smart-connections:2460
await in eval (async)
ensure_embeddings @ plugin:smart-connections:2277

from obsidian-smart-connections.

brianpetro avatar brianpetro commented on July 22, 2024

@robwheatley, thanks for the update! Seems like we're at least making some progress 😊

Please toggle on this option:

Screenshot 2024-02-27 at 6 13 25 PM

It makes the logs provide useful line numbers (the other ones are based on a compiled file).

I just made an update (v2.0.134) to log the blocks causing the above error. This update should give us a better idea of what's going on with that error by letting us see the blocks that are causing it.

Thanks for your help
🌴

from obsidian-smart-connections.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.