Giter VIP home page Giter VIP logo

Comments (9)

rvndbalaji avatar rvndbalaji commented on August 17, 2024 2

@shotarok @lkuligin
Anthropic made tool-use now generally available in their latest version as well as in VertexAI
Can bind_tools and with_structured_output be made available via langchain now?

https://www.anthropic.com/news/tool-use-ga
https://github.com/anthropics/anthropic-sdk-python/releases/tag/v0.27.0

from langchain-google.

lkuligin avatar lkuligin commented on August 17, 2024 1

yes, until tools are supported we should rely on a model following the prompt.
you can even try to add an optional extra step to the chain to convert the output to a desired format like here:

from langchain-google.

rvndbalaji avatar rvndbalaji commented on August 17, 2024 1

@shotarok and @lkuligin I've made the required changes and opened a PR - #288

This is my first contribution, so please do let me know if i've missed anything

from langchain-google.

lkuligin avatar lkuligin commented on August 17, 2024

would you like to work on this one?

from langchain-google.

shotarok avatar shotarok commented on August 17, 2024

Yes, I'm happy to contribute to this project! I plan to use PydanticOutputParser and JsonOutputParser because ChatAnthropicVertex doesn't support function_calling.

from langchain-google.

lkuligin avatar lkuligin commented on August 17, 2024

we should probably start with adding function calling support first.

from langchain-google.

shotarok avatar shotarok commented on August 17, 2024

Oh, I had assumed Claude API on Vertex AI doesn't support function calling since the doc doesn't mention it, but I'll test a function call. I agree. If it works, supporting function call would be the first task.

from langchain-google.

shotarok avatar shotarok commented on August 17, 2024

I'm afraid I couldn't confirm that Claude on Vertex AI supports function calling. I got this error: "tools: Extra inputs are not permitted" with claude-3-haiku@20240307 on Vertex AI. So, I'll create a PR with PydanticOutputParser and JsonOutputParser . I attached the succeeded and failed curl commands, just in case.

A succeeded curl without function calling
$ curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json; charset=utf-8" \
  -d '{
    "anthropic_version": "vertex-2023-10-16",
    "max_tokens": 100,
    "messages": [{"role": "user", "content": "What is the weather like in San Francisco?"}]
}' \
"https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models/${MODEL}:rawPredict"
> {"id":"msg_vrtx_01RYFRNxnQZRDsp71Jq1aKXH","type":"message","role":"assistant","model":"claude-3-haiku-20240307","stop_sequence":null,"usage":{"input_tokens":16,"output_tokens":100},"content":[{"type":"text","text":"The weather in San Francisco can be quite variable, but here's a general overview:\n\n- Mild Temperatures - San Francisco has a Mediterranean climate, with average highs around 65°F (18°C) year-round. Summers are cool, with highs in the 60s or 70s Fahrenheit (15-25°C). Winters are mild, with highs in the 50s and 60s Fahrenheit (10-20"}],"stop_reason":"max_tokens"}
A failed curl with function calling
$ curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json; charset=utf-8" \
  -d '{
    "anthropic_version": "vertex-2023-10-16",
    "max_tokens": 100,
    "tools": [{
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "input_schema": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "The unit of temperature, either \"celsius\" or \"fahrenheit\""
                }
            },
            "required": ["location"]
        }
    }],"messages": [{"role": "user", "content": "What is the weather like in San Francisco?"}]
}' \
"https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models/${MODEL}:rawPredict"
> {"type":"error","error":{"type":"invalid_request_error","message":"tools: Extra inputs are not permitted"}}

from langchain-google.

shotarok avatar shotarok commented on August 17, 2024

After reading langchain-core, I understand the other chat models using PydanticOutputParser and JsonOutputParser assume the LLM API can control output format as JSON like the JSON mode in OpenAI.

Until Claude on Vertex AI supports tools, we may be unable to support with_structured_output in ChatAnthropicVertex. Instead, we can use StructuredOutputParser with giving a prompt by ourselves 1.

Please let me know if you have a different implementation idea. Thanks!

from langchain-google.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.