Self explanatory :p

For easier usage
The Azure OpenAi package has this too

I've been looking at the code for Ollama and working with its console for a bit and trying to get my head wrapped around how to perform the .NET equivalent of Ctrl+C in the API. Is there such a command?

The goal that I'm seeking is to have the ability to "stop" the current stream of text for a given response so that it can then be ready to take in another request from the API.

This is that use case where the user is like "oh wait this is the wrong way to go" and wants to shut off the response so a new message can be sent. The optimal scenario would be a cancellation token or a specific command that can be sent to shut down the current response. I have thought about simply shutting down the current connection and re-connecting but that seems pretty inelegant.

I like the stream response. I most definitely want to use it. Any thoughts on "shunting" or "stopping" a stream? Thanks!

See here for available options: https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values

The options are currently just a string, which is... not optimal
It should rather be a class which has all available options.
So you can set them easier, and then only the ones which arent null get parsed into the request

It seems that options are still a string, making any request with those set invalid.

Hi,the project is very good ,but the method name that return Task type not end with 'Async', i think end with 'Async' is more consistent with common naming conventions, this is just my suggestion 😄

microsoft/semantic-kernel prepare for Update their Ollama Client in Ollama Connector to use OllamaSharp

microsoft/semantic-kernel#5993

but OllamaSharp current TargetFramework is net5.0 and microsoft/semantic-kernel is netstandard2.0.

please change OllamaSharp TargetFramework to netstandard2.0, so microsoft/semantic-kernel can use this package

Hi, I'm missing the format parameter for JSON mode in the GenerateCompletionRequest class.

DOC: https://github.com/ollama/ollama/blob/main/docs/api.md

I'm using JSON mode for long running request, 1 request will take up to 5 minutes.
Is there a way to set the timeout?

Hi Team,

We are working on .Net Framework 4.7.2. Any plan for supporting this?

Thanks,
Rajesh K.

I encountered the following problem: I wanted to send a request to Ollama with some model parameters (mirostat, temperature, seed, etc.). For this, I used the RequestOptions model. However, when sending the request, the Ollama server crashes with a 500 Internal Server Error.

As it turned out, in the RequestOptions model, the Stop parameter is represented as string?, but the Ollama REST API expects an array of strings for this parameter, not just a single string (this can be seen in the official documentation https://github.com/ollama/ollama/blob/main/docs/api.md). Hence, the 500 Internal Server Error.

I was able to fix this problem locally by creating a custom JsonConverter that splits the original string into substrings by a certain delimiter and serializes the required JSON array of strings. Then I just added the attribute [JsonConverter(typeof(StopOptionJsonConverter))] to the Stop property. And this worked for me.

Would you be willing to expose a health check operation? It could return Task<bool> or Task<string> from the base URL as noted on in Ollama issue 1378. I guess I could just do something such as query the list of models or directly access the HttpClient on my own, but having a simple healthcheck operation would be really handy to integrate with ASP.Net Health Checks

Readme should update, thx.

Hi, there
I just found some changes while checking https://github.com/ollama/ollama/blob/main/api/types.go and they are look like:

	// Name is deprecated, see Model
	Name string `json:"name"`

should we also make some changes to align with upstream?

(btw, I'm not a native English speaker, so sorry for my English if there are some mistakes or weirdness)

Please support embedding models too

See the ChatRole class in the Azure OpenAi package

There its basically an Enum which you can put in, and then internally when its evaluated, its just being converted back to a string.

This makes the usage on the dev side much easier, since you dont have to google which ones are available first...

See here: https://github.com/jmorganca/ollama/blob/main/docs/api.md#generate-a-chat-completion
For the available roles...

It should use api/generate, not /api/generate.
It doesn't work if ollama has a base path (like http://ollamaserver/ollama/api/generate), which is common with reverse proxies.

I am building an agentic workflow with Blazor and things were fine using the Ollama.StreamCompletion(), Thing is. I want to give my agents some roles and responsibilities.
The only way you allowed this is by using Ollama.Chat()
When I tried to parse out the message object that is returned and write it out. Which took a lot of effort things were not as smooth as Ollama.StreamCompletion as it contained both my Question and the AI answer, and I could not get clear stream.
I also cannot save the context offline on my PC. because before I used System.Text.Json to store an offline Json file of the context in order to save the chat history. so I do not boot up my agents and build them from scratch every time I start the system.
So, for me there is one of 2 fixes that could work.
1- Access both the context and the StreamCompletion from Ollama.Chat() So I can build my agents and stream their responses.
2- Have a way to assign roles or provide the Message object to the Ollama.StreamCompletion() because it already streams responses and store the context.

After taking a look at your code. I am sure that the 2nd option is the easiest.
My code using Chat

Action<ChatResponseStream> streamer = async stream =>
{
    stream.Message = new()
    {
        Role = ChatRole.System,
        Content = JrPrompt
    };
};
var JuniorChat =   Ollama.Chat(streamer);

My code using Stream Completion

 JunoirContext = await Ollama.StreamCompletion(JrPrompt, JunoirContext, async stream => Console.Write(stream.Response));

the example

// set up the client
var uri = new Uri("http://localhost:11434");
var ollama = new OllamaApiClient(uri);

// list all local models
var models = await ollama.ListLocalModels();

// stream a completion and write to the console
// keep reusing the context to keep the chat topic going
var context = await ollama.StreamCompletion("How are you today?", "llama2", context, stream => Console.WriteLine(stream.Response));

// pull a model and report progress
await ollama.PullModel("mistral", status => Console.WriteLine($"({status.Percent}%) {status.Status}"));

produces :
Cannot use local variable 'context' before it is declared

How to connect with the knowledge base

Version: OllamaSharp 1.0.2
The GenerateEmbeddingResponse doesn't match the actual returned json format from endpoint api/embeddings

I am using OllamaSharp 1.0.2 to call endpoint api/embeddings with model(llama2:70b) and prompt. However, the returned json is { "embedding": [ 1232321, 1232321, ...] } instead of GenerateEmbeddingResponse with { "embeddings": [ 1232321, 1232321, ...] }

According to the official doc, the returned key word should be embedding without s.

You might want to add a note to the readme for others that sit behind a firewall that are experiencing the challenge of Proxy Service. HTTPClient under the covers supports the environment variable NO_PROXY

Adding this to my code gets it working otherwise it's very difficult to diagnose.
System.Environment.SetEnvironmentVariable("NO_PROXY", "localhost");

You can close this issue.. I just wanted to share it with you as I struggled for an hour this morning trying to determine why the code did not work from my office.

Please add summaries to the IOllamaApiClient interface's methods and properties. :)

Currently the Messages property in the ChatRequest class is just a IEnumerable
This makes it "annoying" to add another message
It would make it easier if it was something like a IList
Coming from the Azure OpenAi packages, there the Messages are a IList too, with which you can just use .Add()

0.1.23 adds keep_alive support: https://github.com/ollama/ollama/releases/tag/v0.1.23

Hello! I hope you are well. Really cool useful wrapper for Ollama, we use it now that Ollama is available on Windows. I wanted to know if there is a way to know when the completion has ended (no more tokens generated) with the Chat class.

public class OllamaClient
{
    private const string OllamaApiUri = "http://localhost:11434";
    private const string OllamaModel = "DaVinci-v1:latest";

    private readonly OllamaApiClient _ollama = new(new Uri(OllamaApiUri))
    {
        SelectedModel = OllamaModel
    };

    private Chat? _chat;

    public async Task Setup(Action<ChatResponseStream> streamer)
    {
        var models = await _ollama.ListLocalModels();
        Console.WriteLine("Found the following available models : ");
        foreach (var model in models) Console.WriteLine(model.Name);

        _chat = _ollama.Chat(streamer);
    }

    /// <summary>
    /// Asks the model to generate a completion based on the input
    /// </summary>
    public async Task PerformInference(ChatRole chatRole, string input)
    {
        await _chat.SendAs(chatRole, input);
    }
    
    // How do we know when the inference is over?
}

I'd like to be able to perform some code when the completion is over.

Is there a way to know when the completion has completed?

if queries are long enough the client will timeout before the actual result is retrieved:

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at OllamaApiClient.GenerateCompletion(GenerateCompletionRequest generateRequest, IResponseStreamer`1 streamer)
   at OllamaApiClient.GetCompletion(String prompt, String model, ConversationContext context)
   at llamaAssistance.AiResponseGenerator.GenerateResponse(String query) in /home/neo/Projects/Personal/masstransit/llamaAssistance/llamaAssistance/AiResponseGenerator.cs:line 17
   at Program.<Main>$(String[] args) in /home/neo/Projects/Personal/masstransit/llamaAssistance/llamaAssistance/Program.cs:line 9
   at Program.<Main>(String[] args)

On the Ollama API Documentation they show an example payload that includes a details section that isn't present on the C# response model. Additionally, they make note of an optional system property that may contain a modelfile system prompt.

{
  "modelfile": "# Modelfile generated by \"ollama show\"\n# To build a new Modelfile based on this one, replace the FROM line with:\n# FROM llava:latest\n\nFROM /Users/matt/.ollama/models/blobs/sha256:200765e1283640ffbd013184bf496e261032fa75b99498a9613be4e94d63ad52\nTEMPLATE \"\"\"{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: \"\"\"\nPARAMETER num_ctx 4096\nPARAMETER stop \"\u003c/s\u003e\"\nPARAMETER stop \"USER:\"\nPARAMETER stop \"ASSISTANT:\"",
  "parameters": "num_ctx                        4096\nstop                           \u003c/s\u003e\nstop                           USER:\nstop                           ASSISTANT:",
  "template": "{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: ",
  "details": {
    "format": "gguf",
    "family": "llama",
    "families": ["llama", "clip"],
    "parameter_size": "7B",
    "quantization_level": "Q4_0"
  }
}

These properties are missing from the current ShowModelResponse.