Continue doesn't seem to be getting context of highlighted code when using /edit or /comment about continue HOT 9 CLOSED

Hi @sestinj Thanks. I'm using the default phind-code-34b-v2 (https://huggingface.co/Phind/Phind-CodeLlama-34B-v2). Let me know if there's any testing I can do on my side to help. Happy to contribute what I can.

from continue.

Okay, I've made some progress. This isn't using prompt_templates like you mentioned, but I am using template_messages. I found this in the docs, but the path to template_alpaca_messages has changed from what it says in the docs - https://continue.dev/docs/customization/models#customizing-the-chat-template

I added the following to my config.py

from continuedev.src.continuedev.libs.llm.prompts.chat import template_alpaca_messages

    models=Models(
        default=OpenAI(
                model="Phind-CodeLlama-34B-v2",
                api_base='http://localhost:8000/v1',
                context_length=4096,
                timeout=5000,
                template_messages=template_alpaca_messages,
                # prompt_templates={},
                api_key=""
            ),
    ),

This seems to be formatting things in the chat much better -

from continue.

@brandonbiggs yes, there is a good default prompt_template, so I would fully expect it to work. Does this seem to clear up the last of these issues, or any other quirks you're seeing?

from continue.

I think we're in good shape. Thank you so much for the help!

from continue.

@brandonbiggs could you share which model you are using and I'll play around with the prompts a bit? Based on the [TITLE] in the session header, I would guess some variant of llama/codellama - I've seen a few issues with these understanding the /edit prompt

from continue.

If you do want to play around at all, you'll notice in ~/.continue/config.py that there's a prompt_templates={"edit": "..."} property. This is the thing you can tweak to alter the edit prompt. If it's just a matter of the prompt, then this is exactly what I'll be attempting. If you find that it consistently seems unaware of the code at all, then we might have another problem : )

from continue.

What version of continue are you using? My prompt_templates dict has been empty?

    models=Models(
    #     default=OpenAIFreeTrial(api_key="", model="gpt-4"),
    #     summarize=OpenAIFreeTrial(api_key="", model="gpt-3.5-turbo")
    default=OpenAI(
			model="Phind-CodeLlama-34B-v2",
            api_base='http://localhost:8000/v1',
            context_length=4096,
            timeout=5000,
			prompt_templates={},
			api_key=""
		),
    ),
    ```

from continue.

Ok this actually explains the problem. We don't set a default prompt_template or template_messages for the OpenAI or OpenAIFreeTrial classes because by default they use gpt-4, which doesn't need these. But this obviously doesn't work for other models.

For the prompt_templates, you can change it to {"edit": simplest_edit_prompt} where simplest_edit_prompt is imported like: from continuedev.src.continuedev.libs.llm.prompts.edit import simplest_edit_prompt. And then this should make edits also work well

from continue.

Thanks @sestinj . It appears my /edit as well as /comment are working really well without the prompt_templates, once I just had template_messages configured. Does it make sense to you that it should be working without prompt_templates?

from continue.