No speedup. MacBook Pro 13, M1, 16GB, Ollama, orca-mini.

Local GPT assistance for maximum privacy and offline access.
The plugin allows you to open a context menu on selected text to pick an AI-assistant's action.

Also works with images

No speedup. MacBook Pro 13, M1, 16GB, Ollama, bakllava.

Default actions:

• Continue writing
• Summarize text
• Fix spelling and grammar
• Find action items in text
• General help (just use selected text as a prompt for any purpose)

You can also add yours, share the best actions or get one from the community.

Supported AI Providers:

• Ollama
• OpenAI compatible server (also OpenAI)

This plugin is available in the Obsidian community plugin store https://obsidian.md/plugins?id=local-gpt

You can also install this plugin via BRAT: pfrankov/obsidian-local-gpt

1. Install Ollama. No Windows support yet.
2. Install orca-mini (default) ollama pull orca-mini or any preferred model from the library.

Additional: if you want to enable streaming completion with Ollama you should run it in API-mode: OLLAMA_ORIGINS='*' ollama serve.

There are several options to run local OpenAI-like server:

• llama.cpp
• llama-cpp-python
• LocalAI
• Obabooga Text generation web UI
• LM Studio
• ...maybe more

Here is an example for llama.cpp:

1. Install llama.cpp and follow build instructions for your OS
2. Download a model trained on the ChatML dialog format. For example, Mixtral 8X7B (Dolphin 2.5 version)
3. Run the server by calling ./server -c 4096 --host 0.0.0.0 -t 16 --mlock -m models/dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf or as described in the documentation.

1. Open Obsidian Settings
2. Go to Hotkeys
3. Filter "Local" and you should see "Local GPT: Show context menu"
4. Click on + icon and press hotkey (e.g. ⌘ + M)

It is also possible to specify a fallback to handle requests — this allows you to use larger models when you are online and smaller ones when offline.

Since you can provide any OpenAI-like server, it is possible to use OpenAI servers themselves.
Despite the ease of configuration, I do not recommend this method, since the main purpose of the plugin is to work with private LLMs.

1. Select OpenAI compatible server in Selected AI provider
2. Set OpenAI compatible server URL to https://api.openai.com
3. Retrieve and paste your API key from the API keys page
4. Click "refresh" button and select the model that suits your needs (e.g. gpt-3.5-turbo)

Example screenshot

• Ability to select models from the list instead of typing their names
• Ability to share and apply presets (system prompt + prompt + model)
• Additional AI providers (OpenAI, etc...)
• Streaming completions
• Changing order of the actions
• Editing actions
• Fallback for action if first URL is unavailable (remote GPU)
• Support multimodal models like Llava
• Optional settings for prompts (top_p, top_k, temperature, repeat_penalty)
• Accounting your local documents in results as described here https://ollama.ai/blog/llms-in-obsidian

If you would like to use other providers, please let me know in the discussions.

• Colored Tags that colorizes tags in distinguishable colors.

• Obsidian Ollama.