No speedup. MacBook Pro 13, M1, 16GB, Ollama, orca-mini.
Local GPT assistance for maximum privacy and offline access.
The plugin allows you to open a context menu on selected text to pick an AI-assistant's action.
Also works with images
No speedup. MacBook Pro 13, M1, 16GB, Ollama, bakllava.
Default actions:
- Continue writing
- Summarize text
- Fix spelling and grammar
- Find action items in text
- General help (just use selected text as a prompt for any purpose)
You can also add yours, share the best actions or get one from the community.
Supported AI Providers:
- Ollama
- OpenAI compatible server (also OpenAI)
![Settings](https://private-user-images.githubusercontent.com/584632/296392497-6ab2d802-13ed-42be-aab1-6a3f689b18a0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM5Nzg0MTAsIm5iZiI6MTcyMzk3ODExMCwicGF0aCI6Ii81ODQ2MzIvMjk2MzkyNDk3LTZhYjJkODAyLTEzZWQtNDJiZS1hYWIxLTZhM2Y2ODliMThhMC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODE4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgxOFQxMDQ4MzBaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02MWY1Y2FjMWU4ZWU4MGYzMTczM2UyMTVlNjA4ZWFjYWFlN2JlZTdkNzY0YzVjNmIyYjhkOWY0YmI5ZmMyNDI0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.3e3FgwcVHFWcGN8pZJZyRQrwA69OzwmnIXfs3lZvYUQ)
This plugin is available in the Obsidian community plugin store https://obsidian.md/plugins?id=local-gpt
You can also install this plugin via BRAT: pfrankov/obsidian-local-gpt
- Install Ollama. No Windows support yet.
- Install orca-mini (default)
ollama pull orca-mini
or any preferred model from the library.
Additional: if you want to enable streaming completion with Ollama you should run it in API-mode: OLLAMA_ORIGINS='*' ollama serve
.
There are several options to run local OpenAI-like server:
- llama.cpp
- llama-cpp-python
- LocalAI
- Obabooga Text generation web UI
- LM Studio
- ...maybe more
Here is an example for llama.cpp:
- Install llama.cpp and follow build instructions for your OS
- Download a model trained on the ChatML dialog format. For example, Mixtral 8X7B (Dolphin 2.5 version)
- Run the server by calling
./server -c 4096 --host 0.0.0.0 -t 16 --mlock -m models/dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf
or as described in the documentation.
- Open Obsidian Settings
- Go to Hotkeys
- Filter "Local" and you should see "Local GPT: Show context menu"
- Click on
+
icon and press hotkey (e.g.โ + M
)
It is also possible to specify a fallback to handle requests โ this allows you to use larger models when you are online and smaller ones when offline.
Example video
Kapture.2024-01-11.at.22.16.52.mp4
Since you can provide any OpenAI-like server, it is possible to use OpenAI servers themselves.
Despite the ease of configuration, I do not recommend this method, since the main purpose of the plugin is to work with private LLMs.
- Select
OpenAI compatible server
inSelected AI provider
- Set
OpenAI compatible server URL
tohttps://api.openai.com
- Retrieve and paste your
API key
from the API keys page - Click "refresh" button and select the model that suits your needs (e.g.
gpt-3.5-turbo
)
- Ability to select models from the list instead of typing their names
- Ability to share and apply presets (system prompt + prompt + model)
- Additional AI providers (OpenAI, etc...)
- Streaming completions
- Changing order of the actions
- Editing actions
- Fallback for action if first URL is unavailable (remote GPU)
- Support multimodal models like Llava
- Optional settings for prompts (top_p, top_k, temperature, repeat_penalty)
- Accounting your local documents in results as described here https://ollama.ai/blog/llms-in-obsidian
If you would like to use other providers, please let me know in the discussions.
- Colored Tags that colorizes tags in distinguishable colors.