This thing uses local web server comes with llama-cpp-python to answer question and generate code right inside of the Jupyter Notebook, making new cells.
pip3 install jupyter-gen-ai jupyter nbextension install --python --user jupyter_gen_ai jupyter nbextension enable jupyter_gen_ai/static/main --user
python3 -m llama_cpp.server --n_gpu_layers 0 --n_threads 4 --model models/mistral-7b-instruct-v0.1.Q4_K_M.gguf
Use cmd+enter takes current cell content to GenAI, creates a new cell of the same type below and live feeds the response to this new cell.
- Remove hard-coded url
- Unhardcode model and parameters
- Pass notebook contents in a context
- From a list of urls, fetch them and add it to the context
- Port it to something newer