Doesn't seem to loook at my notes about obsidian-smart-connections HOT 22 OPEN

I turned off block embeddings too and things are working now.

from obsidian-smart-connections.

@brianpetro another issue I noticed. 2 of my notes (Daily and weekly review templates) were not accessible by Obsidian because it is owned by another user (another user in mac). This was resulting in embedding getting stuck at 0.

Added Templates folder in the exclusion and it is working like a charm. I thought I should let you know about the same

from obsidian-smart-connections.

Hey @gramster

Can you send me some screenshots of the developer console and your settings?

Check out this issue for examples #483 (comment)

Thanks for your help in solving this
🌴

from obsidian-smart-connections.

Got the same issue.

version 2.0.122

from obsidian-smart-connections.

Here are my settings

from obsidian-smart-connections.

Thanks @brianlaughlin

Can you also check the developer console?

🌴

from obsidian-smart-connections.

Embeddings settings:

I have an OpenAI key and have selected gpt-4-turbo (128k); everything else is set to default.

If I click on 'Exclude all top level folders', I get this popup,; perhaps it is a clue

After clicking on that I remove my top-level folder from the excluded folders settings, so I am left with ".git, .github, .obsidian, .smart-connections, .trash"

I cleared deveeloper console and ran a query and then this is what I see:

If I quit and restart, I first see:

I get a prompt asking whether I want to embed some 400 documents. After proceeding, I see a boatload of:

Before that I saw a lot of:

from obsidian-smart-connections.

This screenshot indicates things are working

What was the response from the chat to the query you ran?

The errors in the other screenshots are OK.

Lastly, I have issues running the Jina 8K model on my M2 Mac 8GB, so it might be worth trying with a different model. I recommend using the BGE-micro model for both notes/blocks until this gets solved.

from obsidian-smart-connections.

Thanks @brianlaughlin

Can you also check the developer console?

🌴
Here you go. I am happy to run other tests. I'll restart Obsidian and do the same.

from obsidian-smart-connections.

This is onload.

Version was auto updated to:
Looks like it went back on version, ran test still failed. Updated back to 2.0.122.

from obsidian-smart-connections.

Ok, now we're getting somewhere.

It looks like it's failing to retrieve relevant notes.

To help improve the value of the logs, please turn on this setting:

Is the Smart View working?

If it isn't working, this would be a good place to start.

The log line that says "bad escaped character" is probably indicative of what's causing the issues, and this could be causing additional issues, like re-embedding every time you turn on Obsidian/Smart Connections.

A "force refresh" (in the settings) might fix things.

So try toggling on that setting, then restart Obsidian, try a force refresh, observe the logs, try the chat again, and let me know how it goes.

Thanks for your help in solving this!
🌴

from obsidian-smart-connections.

Of course. Yes, I have had several forced refreshes, but I figured you're just working on something. :-) I'll send you what I find out. I'm happy to help.

from obsidian-smart-connections.

@brianpetro I have 32GB RAM; do you still think I should change embeddings library?

I do see this each time I start:

but then when it is done I just see this:

My answers are always starting with things like:

As of my last update in April 2023, I don't have access to your personal notes or any specific, real-time information about ...

I too see this:

might have missed that earlier.

Smart view looks like:

from obsidian-smart-connections.

Did the following:

1. Enabled Debug at Start up.
2. Restarted
3. Did a force refresh
4. When completed ran test again.
5. Test failed.
   Here is the log
   

Smart connections does seem to work on the panel, well it at least shows results. But if I type a ```smart-connections in a note that doesn't work anymore.

from obsidian-smart-connections.

Trying with that micro embeddings model. I get a lot of:

I still saw this at the end:

And I still have the same problem that my notes are not getting consulted. So it is not the embeddings model.

from obsidian-smart-connections.

@brianlaughlin I'm pretty certain that "Bad escaped character" error is the culprit. Are there any files with slashes or quotes in the name? While it shouldn't, they may be causing the issue.

@gramster that should be plenty of RAM for Jina-8k. The truncating notices are expected for BGE-micro. The error suggests that the Note-level embedding is missing for some reason.

It might make sense for both of you to temporarily turn off the block-level embeddings (set to None). Then see how things work after that.

This way, we might be able to narrow down the source of the issue.

Thanks again for helping me figure this out!

🌴

from obsidian-smart-connections.

Thanks @brianpetro
I manually looked and then wrote this program:

import os

def find_files_with_problematic_characters(root_dir):
    problematic_chars = ['<', '>', ':', '"', '/', '\\', '|', '?', '*']  # Added '.' and ' '
    counter = 0  # Initialize counter
    for root, dirs, files in os.walk(root_dir):
        for file_name in files:
            counter += 1  # Increment counter for each file checked
            if any(char in file_name for char in problematic_chars):
                print(f"Found: {file_name}\nLocation: {root}")
            # Display the counter every 100 files
            if counter % 100 == 0:
                print(f"Checked {counter} files...")
    print(f"Total files checked: {counter}")

if __name__ == "__main__":
    directory_to_scan = 'C:\\Users\\brian\\Dropbox\\Obsidian Vault'  # Replace with your directory path
    find_files_with_problematic_characters(directory_to_scan)

Anyone else can use it just change the directory to yours.
It didn't find anything unusual. I'll keep an eye out.

from obsidian-smart-connections.

@brianlaughlin that is very clever; nice work!

Did you disable block-level embeddings (set to "None") and see if the errors still occur?

It could be that the problematic characters occur in headings within notes, and that's why it's the block-level embeddings file that's failing to load.

from obsidian-smart-connections.

@brianpetro
That kinda solved it.
Steps

1. Disabled
2. It redid the embeddings.
3. Restarted - new version found. Upgraded to 2.1
4. Restarted.

Now it's talking to notes again. That's the good part.
It failed the test to find the phone number. That is one of the reasons I decided to use OpenAI's embeddings because it would return better quality results when I tested it against others.

So this problem is resolved. Feel free to ask if there is anything I can do to help improve processes. Thank you for your amazing dedication to a wonderful tool.

from obsidian-smart-connections.

@brianlaughlin 😊

Thanks for your help in getting to the bottom of the issue!

🌴

from obsidian-smart-connections.

@gramster a step in the right direction!

Thanks for following up
🌴

from obsidian-smart-connections.

Same issue here. Like @gramster I also have to turn off block embedding in order for it to work again.

from obsidian-smart-connections.