azure / azure-functions-openai-extension Goto Github PK

View Code? Open in Web Editor NEW

56.0 19.0 18.0 552 KB

An extension that adds support for Azure OpenAI/ OpenAI bindings in Azure Functions for LLM (GPT-3.5-Turbo, GPT-4, etc)

License: MIT License

C# 85.40% Java 14.60%

azure-functions azure-functions-extension azure-openai openai

azure-functions-openai-extension's Issues

Dockerfile for Python functions

Expected Behavior

I'd be nice to be able to run these functions in containers so that we can deploy them to AKS or ACA

What language of preference?

Python

Relevant sample code snipped

No response

Additional Information

No response

Cannot use semantic search bindings in .NET 8

When I change my standard sample demo project to target .NET 8 in the csproj, the project builds clean, but then I get this runtime error when I test a post to IngestData (uses Embeddings binding):

This branch exactly repros it as you use test.http and post to the local IngestData function:.

Functions:

        IngestData: [POST] http://localhost:7071/api/IngestData

        PromptData: [POST] http://localhost:7071/api/PromptData

        WhoIs:  http://localhost:7071/api/whois/{name}

        OpenAI::ChatBotEntity: entityTrigger

For detailed output, run func with --verbose flag.
[2024-02-15T18:38:50.834Z] Host lock lease acquired by instance ID '000000000000000000000000731A3487'.
[2024-02-15T18:38:52.563Z] Executing 'IngestData' (Reason='This function was programmatically called via the host APIs.', Id=e43173fd-3292-44f4-b59c-c519224a4bd8)
[2024-02-15T18:38:52.586Z] Executed 'IngestData' (Failed, Id=e43173fd-3292-44f4-b59c-c519224a4bd8, Duration=39ms)
[2024-02-15T18:38:52.586Z] System.Private.CoreLib: Exception while executing function: IngestData. Microsoft.Azure.WebJobs.Host: One or more errors occurred. (Exception binding parameter 'req') (Exception binding parameter 'embeddings'). Exception binding parameter 'req'. System.Private.CoreLib: Could not load type 'System.Runtime.CompilerServices.NullableContextAttribute' from assembly 'System.Runtime, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

It is throwing on this line of code that uses the EmbeddingsRequest:embeddings record object, and note I have two fields, with RawText being null here:

        public record EmbeddingsRequest(string RawText, string FilePath);

//...
            await output.AddAsync(new SearchableDocument(title, embeddings));

If i changed EmbeddingsRequest definition to match the sample having only a single field in the record, I go back to getting Build errors all the time (maybe this was bound to happen and I forced a rebuild?):

        public record EmbeddingsRequest(string FilePath);
        public record SemanticSearchRequest(string Prompt);

        [FunctionName("IngestData")]
        public static async Task<IActionResult> IngestData(
            [HttpTrigger(AuthorizationLevel.Function, "post")] EmbeddingsRequest req,
            [Embeddings("{FilePath}", inputType: InputType.FilePath, 
            Model = "%AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT%")] EmbeddingsContext embeddings,
            [SemanticSearch("KustoConnectionString", "Documents")] IAsyncCollector<SearchableDocument> output)
        {
            string title = Path.GetFileNameWithoutExtension(req.FilePath);
            await output.AddAsync(new SearchableDocument(title, embeddings));
            return new OkObjectResult(new { status = "success", title, chunks = embeddings.Count });
        }

[2024-02-15T18:34:32.589Z] Found /Users/paulyuk/src/paulyuk/functions/ai/ai-bindings/ai-bindings.csproj. Using for user secrets file configuration.
[2024-02-15T18:34:33.488Z] No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
[2024-02-15T18:34:33.507Z] The 'IngestData' function is in error: The binding type(s) 'embeddings, semanticSearch' are not registered. Please ensure the type is correct and the binding extension is installed.
[2024-02-15T18:34:33.507Z] The 'PromptData' function is in error: The binding type(s) 'semanticSearch' are not registered. Please ensure the type is correct and the binding extension is installed.
[2024-02-15T18:34:33.507Z] The 'WhoIs' function is in error: The binding type(s) 'textCompletion' are not registered. Please ensure the type is correct and the binding extension is installed.
For detailed output, run func with --verbose flag.

The net of this (no pun intended :) ) is I cannot use .NET 8, or figure out a workaround.

Readme for assistants should include creation of testdb database, my-todos container, and id for partition key

In the Readme.md for assistants, https://github.com/Azure/azure-functions-openai-extension/blob/main/samples/assistant/README.md#prerequisites, we should add instructions to create the testdb database, my-todos container, and id for partition key.

[Bug] Semantic Search with Kusto returns unexpected match

What language are you using?

Dotnet (OOP)

Expected Behavior

Index couple of documents in Kusto, I indexed two small documents with chunk size 3 and 8 respectively.

I am asking a very simple question based on document 2.
Kusto should return an embedding match - chunk or snippet which has closest match to my query.

Actual Behavior

It doesn't matter what question I ask, Kusto query always returns me the same chunk.
It goes to OpenAI, since the Kusto snippet doesn't make sense, OpenAI tries to answer from its own knowledge pool and gives a random document name as reference since SystemPrompt asks to give references.

Host.json

{
"version": "2.0",
"logging": {
"logLevel": {
"Microsoft.Azure.WebJobs.Extensions.OpenAI": "Information"
}
},
"extensions": {
"openai": {
"searchProvider": {
"type": "kusto"
}
}
}
}

Steps to Reproduce

Set up local.settings.json for SemanticSearchEmbeddings sample for Kusto
Run the sample
In demo.http, provide two file paths for ingestion API, ingest the documents into Kusto
In demo.http, give a prompt based on document which was ingested later
Have breakpoints to check Kusto response
Also, observe the final response from the API.

Relevant code being tried

No response

Relevant log output

No response

Where are you facing this problem?

Local - Core Tools

Additional Information

No response

[Bug] Example AssistantSample Skill (AddToDo) repetitively executes

What language are you using?

Dotnet (OOP)

Expected Behavior

When running the AssistantSample I expected that when a given (AddToDo) skill was identified, executed and executed successfully it would add a single entry to the history for the execution of the skill.

Actual Behavior

The AssistantSample kept identifying the (AddToDo) skill and executing it over and over again. This was seen both in the chat history and the Azure Function Runtime console.

Host.json

{
"version": "2.0",
"logging": {
"logLevel": {
"Microsoft.Azure.WebJobs.Extensions.OpenAI": "Information"
}
},
"extensions": {
"openai": {
"storageConnectionName": "AzureWebJobsStorage",
"collectionName": "SampleChatState"
}
}
}

Steps to Reproduce

Cloned Repo and run the AssistantSample as part of the examples.

Configured Open AI settings:

Model Name: gpt-4
Model Version: 1106-Preview

Using the demo.http file:

PUT http://localhost:7071/api/assistants/assistant123
Returns 201 Created:

{
  "assistantId": "assistant123"
}

GET http://localhost:7071/api/assistants/assistant123?timestampUTC=2023-01-01T00:00:00Z
Returns 200 OK:

{
  "id": "assistant123",
  "exists": true,
  "createdAt": "2024-05-30T14:51:51.3586337Z",
  "lastUpdatedAt": "2024-05-30T14:51:51.3586602Z",
  "totalMessages": 1,
  "totalTokens": 0,
  "recentMessages": [
    {
      "content": "Don't make assumptions about what values to plug into functions.\r\nAsk for clarification if a user request is ambiguous.",
      "role": "system"
    }
  ]
}

POST http://localhost:7071/api/assistants/assistant123?message=Oh,%20and%20to%20take%20out%20the%20trash
Returns 200 OK:

The task to take out the trash has been added to your to-do list.

GET http://localhost:7071/api/assistants/assistant123?timestampUTC=2023-01-01T00:00:00Z
Returns 200 OK:

{
  "id": "assistant123",
  "exists": true,
  "createdAt": "2024-05-30T14:51:51.3586337Z",
  "lastUpdatedAt": "2024-05-30T14:55:32.1207959Z",
  "totalMessages": 12,
  "totalTokens": 327,
  "recentMessages": [
    {
      "content": "Don't make assumptions about what values to plug into functions.\r\nAsk for clarification if a user request is ambiguous.",
      "role": "system"
    },
    {
      "content": "Oh, and to take out the trash",
      "role": "user"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The function call succeeded. Let the user know that you completed the action.",
      "role": "function",
      "name": "AddTodo"
    },
    {
      "content": "The task to take out the trash has been added to your to-do list.",
      "role": "assistant"
    }
  ]
}

Looking at the chat history you can see that it appears that the skill has been executed successfully but it has been repeated more times than it should have.

Relevant code being tried

[Function(nameof(AddTodo))]
    public Task AddTodo([AssistantSkillTrigger("Create a new todo task")] string taskDescription)
    {
        if (string.IsNullOrEmpty(taskDescription))
        {
            throw new ArgumentException("Task description cannot be empty");
        }

        this.logger.LogInformation("Adding todo: {task}", taskDescription);

        string todoId = Guid.NewGuid().ToString()[..6];
        return this.todoManager.AddTodoAsync(new TodoItem(todoId, taskDescription));
    }

    [Function(nameof(PostUserQuery))]
    public static async Task<HttpResponseData> PostUserQuery(
        [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "assistants/{assistantId}")] HttpRequestData req,
        string assistantId,
        [AssistantPostInput("{assistantId}", "{Query.message}", Model = "%CHAT_MODEL_DEPLOYMENT_NAME%")] AssistantState state)
    {
        HttpResponseData response = req.CreateResponse(HttpStatusCode.OK);
        response.Headers.Add("Content-Type", "text/plain");
        await response.WriteStringAsync(state.RecentMessages.LastOrDefault()?.Content ?? "No response returned.");
        return response;
    }

Relevant log output

[2024-05-30T14:51:45.927Z] Host lock lease acquired by instance ID '0000000000000000000000009540DA93'.
[2024-05-30T14:51:50.814Z] Executing 'Functions.CreateAssistant' (Reason='This function was programmatically called via the host APIs.', Id=c2bcf30f-5fec-4c31-914b-bd2c67de484d)
[2024-05-30T14:51:51.131Z] [assistant123] Creating new chat session with instructions = "Don't make assumptions about what values to plug into functions.
[2024-05-30T14:51:51.136Z] Ask for clarification if a user request is ambiguous."
[2024-05-30T14:51:51.297Z] Deleting 14 record(s) for assistant 'assistant123'.
[2024-05-30T14:51:51.386Z] Created assistant 'assistant123'
[2024-05-30T14:51:51.415Z] Executed 'Functions.CreateAssistant' (Succeeded, Id=c2bcf30f-5fec-4c31-914b-bd2c67de484d, Duration=630ms)
[2024-05-30T14:53:42.116Z] Executing 'Functions.GetChatState' (Reason='This function was programmatically called via the host APIs.', Id=60682119-ac28-415d-9f05-4c9b8c83f286)
[2024-05-30T14:53:42.159Z] Reading state for assistant entity 'assistant123' and getting chat messages after 2023-01-01T00:00:00.0000000Z
[2024-05-30T14:53:42.182Z] Returning 1/1 chat messages from entity 'assistant123'
[2024-05-30T14:53:42.280Z] Executed 'Functions.GetChatState' (Succeeded, Id=60682119-ac28-415d-9f05-4c9b8c83f286, Duration=165ms)
[2024-05-30T14:54:59.874Z] Executing 'Functions.PostUserQuery' (Reason='This function was programmatically called via the host APIs.', Id=10d2f824-4165-40f4-b6cd-916fed757962)
[2024-05-30T14:54:59.881Z] Posting message to assistant 'assistant123': Oh, and to take out the trash
[2024-05-30T14:54:59.900Z] Posting message to assistant entity 'assistant123'
[2024-05-30T14:54:59.911Z] [assistant123] Received message: Oh, and to take out the trash
[2024-05-30T14:55:01.481Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:01.489Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:01.497Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=1859be89-675c-4baf-b184-eb7ccfc4b144)
[2024-05-30T14:55:01.505Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:07.324Z] Adding todo ID = 8091ab to container 'myTasks'.[2024-05-30T14:55:07.324Z] Adding todo: Take out the trash

[2024-05-30T14:55:07.872Z] Executed 'Functions.AddTodo' (Succeeded, Id=1859be89-675c-4baf-b184-eb7ccfc4b144, Duration=6376ms)
[2024-05-30T14:55:07.882Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:11.553Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:11.557Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:11.561Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=8a9b0dee-dc8a-46ca-9de6-912e632bb726)
[2024-05-30T14:55:11.564Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:11.596Z] Adding todo: Take out the trash
[2024-05-30T14:55:11.597Z] Adding todo ID = a9e34f to container 'myTasks'.
[2024-05-30T14:55:11.617Z] Executed 'Functions.AddTodo' (Succeeded, Id=8a9b0dee-dc8a-46ca-9de6-912e632bb726, Duration=56ms)
[2024-05-30T14:55:11.627Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:15.306Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:15.310Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:15.312Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=255f169a-dfe5-4ad5-a73e-f3dff983629f)
[2024-05-30T14:55:15.313Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:15.346Z] Adding todo: Take out the trash
[2024-05-30T14:55:15.346Z] Adding todo ID = f919f5 to container 'myTasks'.
[2024-05-30T14:55:15.366Z] Executed 'Functions.AddTodo' (Succeeded, Id=255f169a-dfe5-4ad5-a73e-f3dff983629f, Duration=54ms)
[2024-05-30T14:55:15.369Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:18.193Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:18.197Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:18.200Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=31ba1b22-94ca-405c-a232-a61e7454c459)
[2024-05-30T14:55:18.203Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:18.241Z] Adding todo: Take out the trash
[2024-05-30T14:55:18.242Z] Adding todo ID = 46d7f8 to container 'myTasks'.
[2024-05-30T14:55:18.260Z] Executed 'Functions.AddTodo' (Succeeded, Id=31ba1b22-94ca-405c-a232-a61e7454c459, Duration=60ms)
[2024-05-30T14:55:18.269Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:20.519Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:20.522Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:20.526Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=c4ef43e0-66e2-4367-9311-29a0e94719a7)
[2024-05-30T14:55:20.528Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:20.569Z] Adding todo: Take out the trash
[2024-05-30T14:55:20.570Z] Adding todo ID = 6979b1 to container 'myTasks'.
[2024-05-30T14:55:20.581Z] Executed 'Functions.AddTodo' (Succeeded, Id=c4ef43e0-66e2-4367-9311-29a0e94719a7, Duration=55ms)
[2024-05-30T14:55:20.586Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:22.369Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:22.373Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"take out the trash"}
[2024-05-30T14:55:22.376Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=08df611e-2ab6-4bc0-98a1-361de7d6c4c0)
[2024-05-30T14:55:22.378Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:22.411Z] Adding todo: take out the trash
[2024-05-30T14:55:22.412Z] Adding todo ID = 2291cc to container 'myTasks'.
[2024-05-30T14:55:22.430Z] Executed 'Functions.AddTodo' (Succeeded, Id=08df611e-2ab6-4bc0-98a1-361de7d6c4c0, Duration=53ms)
[2024-05-30T14:55:22.434Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:24.260Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:24.263Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:24.267Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=d1a7d5ff-9cc8-4772-8bf0-b5b3056dcf0b)
[2024-05-30T14:55:24.271Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:24.310Z] Adding todo: Take out the trash[2024-05-30T14:55:24.311Z] Adding todo ID = 48e056 to container 'myTasks'.

[2024-05-30T14:55:24.325Z] Executed 'Functions.AddTodo' (Succeeded, Id=d1a7d5ff-9cc8-4772-8bf0-b5b3056dcf0b, Duration=57ms)
[2024-05-30T14:55:24.329Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:27.770Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:27.774Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:27.777Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=a0565310-c569-4619-b4fd-ea6d9aed0de2)
[2024-05-30T14:55:27.779Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:27.810Z] Adding todo: Take out the trash
[2024-05-30T14:55:27.811Z] Adding todo ID = 2a861c to container 'myTasks'.
[2024-05-30T14:55:27.831Z] Executed 'Functions.AddTodo' (Succeeded, Id=a0565310-c569-4619-b4fd-ea6d9aed0de2, Duration=53ms)
[2024-05-30T14:55:27.837Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:30.592Z] [assistant123] Found 1 function call(s) in response
[2024-05-30T14:55:30.595Z] [assistant123] Calling function 'AddTodo' with arguments: {"taskDescription":"Take out the trash"}
[2024-05-30T14:55:30.597Z] Executing 'Functions.AddTodo' (Reason='(null)', Id=c404eefc-b134-4940-a215-77b33f74212e)
[2024-05-30T14:55:30.599Z] Invoking user-code function 'AddTodo'
[2024-05-30T14:55:30.634Z] Adding todo: Take out the trash
[2024-05-30T14:55:30.634Z] Adding todo ID = 5bc91f to container 'myTasks'.
[2024-05-30T14:55:30.649Z] Executed 'Functions.AddTodo' (Succeeded, Id=c404eefc-b134-4940-a215-77b33f74212e, Duration=52ms)
[2024-05-30T14:55:30.655Z] [assistant123] Function 'AddTodo' returned the following content: (null)
[2024-05-30T14:55:32.114Z] [assistant123] Got LLM response consisting of 17 tokens: The task to take out the trash has been added to your to-do list.
[2024-05-30T14:55:32.118Z] [assistant123] Chat length is now 12 messages
[2024-05-30T14:55:32.133Z] Returning 1/12 chat messages from entity 'assistant123'
[2024-05-30T14:55:32.177Z] Executed 'Functions.PostUserQuery' (Succeeded, Id=10d2f824-4165-40f4-b6cd-916fed757962, Duration=32303ms)

Where are you facing this problem?

Local - Core Tools

Additional Information

I have tried to build out other skills where it just returns a string as a result, however it too gets called over and over. My concern would be that if the skill is calling out to a REST service or something which is rate-limited or charged based on number of calls with an API call, the repetitive nature of the issue could cause problems.

[Bug] Assistant Functions API not triggerring skills with 0301 GPT 35 model version

What language are you using?

Dotnet (OOP)

Expected Behavior

Assistant Functions should work as demonstrated in the sample, add todo tasks and recognise the skills

Actual Behavior

No task being added to ToDo, errors in the logs (shared)
Not retrieving the tasks.

Host.json

No response

Steps to Reproduce

No response

Relevant code being tried

https://github.com/Azure/azure-functions-openai-extension/tree/main/samples/assistant/csharp-ooproc/AssistantSample

Relevant log output

[2024-03-20T18:49:47.419Z] Executing 'Functions.PostUserQuery' (Reason='This function was programmatically called via the host APIs.', Id=8144aae9-3bb5-40fa-b79f-6200344c742e)
[2024-03-20T18:49:47.434Z] Posting message to assistant 'assistant128': Remind me to call my dad
[2024-03-20T18:49:47.441Z] Posting message to assistant entity 'assistant128'
[2024-03-20T18:49:47.455Z] [assistant128] Received message: Remind me to call my dad
[2024-03-20T18:49:55.468Z] [assistant128] Found 1 function call(s) in response
[2024-03-20T18:49:55.470Z] [assistant128] Calling function 'functions' with arguments: // Assuming this is a request to create a new todo task
// We need clarification on the task details, like when should it be completed, or if there's any additional notes for the task
type AddTodo = (_: {
dueDate?: Date,
notes?: string,
}) => any;
 
// We can also add an optional reminder if necessary, so the assistant can remind the user about the task before the due date
type AddReminder = (_: {
reminderDate: Date,
}) => any;
 
[2024-03-20T18:49:55.476Z] [assistant128] Function 'functions' failed with an unhandled exception
[2024-03-20T18:49:55.476Z] WebJobs.Extensions.OpenAI: No skill registered with name 'functions'.
[2024-03-20T18:49:56.130Z] [assistant128] Got LLM response consisting of 23 tokens: I'm sorry, I didn't understand what action you are requesting. Can you please give me more context?
[2024-03-20T18:49:56.132Z] [assistant128] Chat length is now 4 messages
[2024-03-20T18:49:56.139Z] Executed 'Functions.PostUserQuery' (Succeeded, Id=8144aae9-3bb5-40fa-b79f-6200344c742e, Duration=8722ms)
 
 
[2024-03-20T19:01:24.923Z] Created assistant 'assistant134'
[2024-03-20T19:01:24.924Z] Executed 'Functions.CreateAssistant' (Succeeded, Id=0d978090-5e4c-497f-9d27-06615272d746, Duration=42076ms)
[2024-03-20T19:01:27.261Z] Executing 'Functions.PostUserQuery' (Reason='This function was programmatically called via the host APIs.', Id=b132c6c6-bfb5-42a8-90f1-482d42b3e152)
[2024-03-20T19:01:27.285Z] Posting message to assistant 'assistant134': Remind me to call my dad
[2024-03-20T19:01:27.285Z] Posting message to assistant entity 'assistant134'
[2024-03-20T19:01:27.289Z] [assistant134] Received message: Remind me to call my dad
[2024-03-20T19:01:28.624Z] [assistant134] Got LLM response consisting of 35 tokens: Sure thing! Should I add this as a todo task or just remind you right now? Also, do you want a specific date and time or just a general reminder?
[2024-03-20T19:01:28.625Z] [assistant134] Chat length is now 3 messages
[2024-03-20T19:01:28.628Z] Executed 'Functions.PostUserQuery' (Succeeded, Id=b132c6c6-bfb5-42a8-90f1-482d42b3e152, Duration=1367ms)
[2024-03-20T19:02:13.462Z] Executing 'Functions.PostUserQuery' (Reason='This function was programmatically called via the host APIs.', Id=427db8ae-d455-4fbe-afdc-4db086d590cf)
[2024-03-20T19:02:13.480Z] Posting message to assistant 'assistant134': add as a todo task with a general reminder
[2024-03-20T19:02:13.481Z] Posting message to assistant entity 'assistant134'
[2024-03-20T19:02:13.485Z] [assistant134] Received message: add as a todo task with a general reminder
[2024-03-20T19:02:15.111Z] [assistant134] Found 1 function call(s) in response
[2024-03-20T19:02:15.113Z] [assistant134] Calling function 'python' with arguments: // Assuming there's a function to add a new todo task
functions.addTodo({
  taskDescription: "Call my dad",
});
 
// You can set a reminder for yourself to check your todo list at a specific time, or just set an alarm on your phone to remind you to check your list periodically.
[2024-03-20T19:02:15.186Z] [assistant134] Function 'python' failed with an unhandled exception
[2024-03-20T19:02:15.187Z] WebJobs.Extensions.OpenAI: No skill registered with name 'python'.
[2024-03-20T19:02:15.882Z] [assistant134] Got LLM response consisting of 39 tokens: Oops, it looks like I don't have access to a todo list to add this task to. Do you want me to create a new list for you or do you already have one?
[2024-03-20T19:02:15.885Z] [assistant134] Chat length is now 6 messages
[2024-03-20T19:02:15.892Z] Executed 'Functions.PostUserQuery' (Succeeded, Id=427db8ae-d455-4fbe-afdc-4db086d590cf, Duration=2429ms)

Where are you facing this problem?

Local - Core Tools

Additional Information

Model specs, it works with

Model specs, it doesn't work with

TotalTokens Count after initialization should count instructions tokens.

In the current implementation for TotalTokens in ChatCompletion.

Initialise the chat state with PUT method

PUT http://localhost:7071/api/chats/test123
Content-Type: application/json

{
  "instructions": "You are a helpful chatbot. In all your English responses, speak as if you are Shakespeare."
}

Retrieve the chat state with GET method (TotalTokens is 0) because nothing has been posted to OpenAI yet and it has not returned consumed tokens.
GET http://localhost:7071/api/chats/test123?timestampUTC=2023-08-10T07:51:10Z

Behavior:
Actual: The response returns totalTokens to be 0.
Expected: it should count the instructions tokens.

Action required: migrate or opt-out of migration to GitHub inside Microsoft

Migrate non-Open Source or non-External Collaboration repositories to GitHub inside Microsoft

In order to protect and secure Microsoft, private or internal repositories in GitHub for Open Source which are not related to open source projects or require collaboration with 3rd parties (customer, partners, etc.) must be migrated to GitHub inside Microsoft a.k.a GitHub Enterprise Cloud with Enterprise Managed User (GHEC EMU).

Action

✍️ Please RSVP to opt-in or opt-out of the migration to GitHub inside Microsoft.

❗Only users with admin permission in the repository are allowed to respond. Failure to provide a response will result to your repository getting automatically archived.🔒

Instructions

Reply with a comment on this issue containing one of the following optin or optout command options below.

✅ Opt-in to migrate

@gimsvc optin --date <target_migration_date in mm-dd-yyyy format>

Example: @gimsvc optin --date 03-15-2023

❌ Opt-out of migration

@gimsvc optout --reason <staging|collaboration|delete|other>

Example: @gimsvc optout --reason staging

Options:

staging : This repository will ship as Open Source or go public

collaboration : Used for external or 3rd party collaboration with customers, partners, suppliers, etc.

delete : This repository will be deleted because it is no longer needed.

other : Other reasons not specified

Need more help? 🖐️

Email [email protected]. ✉️
Post your questions in GitHub inside Microsoft Team in Microsoft Teams. 🗨️

[BUG] Assistant Functions - GET assistants call has too many function calls in 1106 model version of GPT

When testing, we sporadically see multiple calls for "The function call succeeded. Let the user know you completed the action", when we should only be seeing one. This is likely caused by the model's response and needs to be investigated further. Here is an example JSON output of what we're seeing:

{
    "id": "assistant678",
    "exists": true,
    "status": null,
    "createdAt": "2024-03-01T17:31:26.849899Z",
    "lastUpdatedAt": "2024-03-01T17:32:22.5504361Z",
    "totalMessages": 6,
    "totalTokens": 173,
    "recentMessages": [
        {
            "content": "Don't make assumptions about what values to plug into functions.\r\nAsk for clarification if a user request is ambiguous.",
            "role": "system"
        },
        {
            "content": "Remind me to call my dad",
            "role": "user"
        },
        {
            "content": "The function call succeeded. Let the user know that you completed the action.",
            "role": "function"
        },
        {
            "content": "The function call succeeded. Let the user know that you completed the action.",
            "role": "function"
        },
        {
            "content": "The function call succeeded. Let the user know that you completed the action.",
            "role": "function"
        },
        {
            "content": "I have added a reminder for you to call your dad.",
            "role": "assistant"
        }
    ]
}

[Bug] Cannot start function with Python

What language are you using?

Python

Expected Behavior

func start to start the function

Actual Behavior

Exception: AttributeError: 'FunctionApp' object has no attribute 'text_completion_input'

Host.json

Same as the example

Steps to Reproduce

Edit the local settings.json
func start
--> Works with a simple HTTP function

Relevant code being tried

No response

Relevant log output

No response

Where are you facing this problem?

Local - Core Tools

Additional Information

No response

dotnet isolated sample creation with http trigger template doesn't return any response and runs until time out.

Function specification from template which should be changed according to samples/textcompletion/csharp-inproc:

[Function("Function1")]
public IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "get", "post")] HttpRequest req)
{
    _logger.LogInformation("C# HTTP trigger function processed a request.");
    return new OkObjectResult("Welcome to Azure Functions!");
}

Program.cs

using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(services =>
    {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
    })
    .Build();

host.Run();

References

    <FrameworkReference Include="Microsoft.AspNetCore.App" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.20.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.1.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore" Version="1.2.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.16.4" />
    <PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.21.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.ApplicationInsights" Version="1.1.0" />

Sample expects the Program.cs to be:

using Microsoft.Extensions.Hosting;


var host = new HostBuilder()
    .ConfigureFunctionsWorkerDefaults()
    .Build();

host.Run();

Add support and samples for URL based input types for Embeddings Binding

I noticed in the recent Nuget samples we have the user write their own record for embeddings requests, and it only handles this one input type:

public record EmbeddingsRequest(string FilePath);

In prior samples we handled 2 of the 3 cases ; - )

public record EmbeddingsRequest(string RawText, string FilePath);

My opinion is we should probably give the user a standard type for embedding request and handle all 3 of the cases: RawText, FilePath, and Url (cool new one!). But at least have sample code that is aware of each case like i have above pls.

Rename "chat bots" to "assistants"

This issue tracks rebranding chat bots as assistants, which is consistent with OpenAI's naming conventions.

As part of this, we should also rename the "Agents" namespace to Assistants ("agent" is a misnomer, and we might decide to use it for something else later).

ASP.NET WhoIs example should use IActionResult

Looking at this example, the code is perfectly fine, just it's not modern ASP.NET like.

[FunctionName(nameof(WhoIs))]
public static string WhoIs(
    [HttpTrigger(AuthorizationLevel.Function, Route = "whois/{name}")] HttpRequest req,
    [TextCompletion("Who is {name}?", Model = "gpt-35-turbo")] TextCompletionResponse response)
{
    return response.Content;
}

Suggest changing this in all places to be this instead:

        [FunctionName(nameof(WhoIs))]
        public static IActionResult WhoIsFunction(
            [HttpTrigger(AuthorizationLevel.Function, Route = "whois/{name}")] HttpRequest req,
            [TextCompletion("Who is {name}?")] TextCompletionResponse response)
        {
            return new OkObjectResult(response.Content);
        }

Feature: Parallel function-calling for AI Assistants

Today, the AI assistant triggers can only be invoked one-at-a-time. However, for performance reasons, it can be useful to support parallel function calling, especially since interacting with OpenAI APIs for advanced models like GPT-4 can be time consuming.

This issue tracks adding support for parallel function-calling, which is already a feature of OpenAI's function calling API.

Remove ability to use Azure OpenAI API keys

Expected Behavior

As part of a product review with executive leadership, an ask was made to remove the ability to use Azure OpenAI API keys. Instead, we should support only managed identity authentication with Azure OpenAI and our Azure-hosted search providers. This ensures that customers are always using the most secure configurations.

Additional Information

Naturally, we'll want to support both system and user-assigned identities. New APIs may be required specifically for configuring user-assigned identities.

Making small changes to a client project using extension leads to errors

When I make changes or come back to my sample that uses our new Nugets, I see these types of errors a lot when I func start:

Example 1

[2024-02-15T18:11:35.534Z] Found /Users/paulyuk/src/paulyuk/functions/ai/ai-bindings/ai-bindings.csproj. Using for user secrets file configuration.
[2024-02-15T18:11:42.982Z] A host error has occurred during startup operation '9cf9f0e1-32ad-4175-8390-d95c53563625'.
[2024-02-15T18:11:42.982Z] Azure.Identity: Method not found: 'Void Azure.Core.ClientOptions..ctor(Azure.Core.DiagnosticsOptions)'.
Value cannot be null. (Parameter 'provider')
[2024-02-15T18:11:43.072Z] Host startup operation has been canceled

Example 2

[2024-02-15T18:12:29.734Z] Found /Users/paulyuk/src/paulyuk/functions/ai/ai-bindings/ai-bindings.csproj. Using for user secrets file configuration.
[2024-02-15T18:12:30.737Z] No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
[2024-02-15T18:12:30.757Z] The 'InngestData' function is in error: The binding type(s) 'embeddings, semanticSearch' are not registered. Please ensure the type is correct and the binding extension is installed.
[2024-02-15T18:12:30.757Z] The 'PromptData' function is in error: The binding type(s) 'semanticSearch' are not registered. Please ensure the type is correct and the binding extension is installed.
[2024-02-15T18:12:30.757Z] The 'WhoIs' function is in error: The binding type(s) 'textCompletion' are not registered. Please ensure the type is correct and the binding extension is installed.
For detailed output, run func with --verbose flag.
[2024-02-15T18:12:36.449Z] Host lock lease acquired by instance ID '000000000000000000000000731A3487'.

Closing my terminal and restarting usually mitigates it and errors go away.

Steps to reproduce:

clone this repo and change to this folder: https://github.com/paulyuk/functions/tree/main/ai/ai-bindings
func start <-- should be ok
change any value in ai-bindings.csproj
func start again <-- error often starts

azure / azure-functions-openai-extension Goto Github PK

azure-functions-openai-extension's Issues

Expected Behavior

What language of preference?

Relevant sample code snipped

Additional Information

What language are you using?

Expected Behavior

Actual Behavior

Host.json

Steps to Reproduce

Relevant code being tried

Relevant log output

Where are you facing this problem?

Additional Information

What language are you using?

Expected Behavior

Actual Behavior

Host.json

Steps to Reproduce

Relevant code being tried

Relevant log output

Where are you facing this problem?

Additional Information

What language are you using?

Expected Behavior

Actual Behavior

Host.json

Steps to Reproduce

Relevant code being tried

Relevant log output

Where are you facing this problem?

Additional Information

Migrate non-Open Source or non-External Collaboration repositories to GitHub inside Microsoft

Action

Instructions

Need more help? 🖐️

What language are you using?

Expected Behavior

Actual Behavior

Host.json

Steps to Reproduce

Relevant code being tried

Relevant log output

Where are you facing this problem?

Additional Information

Expected Behavior

Additional Information

Recommend Projects

Recommend Topics

Recommend Org