Comments (9)
@shotarok @lkuligin
Anthropic made tool-use now generally available in their latest version as well as in VertexAI
Can bind_tools
and with_structured_output
be made available via langchain now?
https://www.anthropic.com/news/tool-use-ga
https://github.com/anthropics/anthropic-sdk-python/releases/tag/v0.27.0
from langchain-google.
yes, until tools are supported we should rely on a model following the prompt.
you can even try to add an optional extra step to the chain to convert the output to a desired format like here:
from langchain-google.
@shotarok and @lkuligin I've made the required changes and opened a PR - #288
This is my first contribution, so please do let me know if i've missed anything
from langchain-google.
would you like to work on this one?
from langchain-google.
Yes, I'm happy to contribute to this project! I plan to use PydanticOutputParser and JsonOutputParser because ChatAnthropicVertex doesn't support function_calling.
from langchain-google.
we should probably start with adding function calling support first.
from langchain-google.
Oh, I had assumed Claude API on Vertex AI doesn't support function calling since the doc doesn't mention it, but I'll test a function call. I agree. If it works, supporting function call would be the first task.
from langchain-google.
I'm afraid I couldn't confirm that Claude on Vertex AI supports function calling. I got this error: "tools: Extra inputs are not permitted"
with claude-3-haiku@20240307
on Vertex AI. So, I'll create a PR with PydanticOutputParser
and JsonOutputParser
. I attached the succeeded and failed curl commands, just in case.
A succeeded curl without function calling
$ curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d '{
"anthropic_version": "vertex-2023-10-16",
"max_tokens": 100,
"messages": [{"role": "user", "content": "What is the weather like in San Francisco?"}]
}' \
"https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models/${MODEL}:rawPredict"
> {"id":"msg_vrtx_01RYFRNxnQZRDsp71Jq1aKXH","type":"message","role":"assistant","model":"claude-3-haiku-20240307","stop_sequence":null,"usage":{"input_tokens":16,"output_tokens":100},"content":[{"type":"text","text":"The weather in San Francisco can be quite variable, but here's a general overview:\n\n- Mild Temperatures - San Francisco has a Mediterranean climate, with average highs around 65°F (18°C) year-round. Summers are cool, with highs in the 60s or 70s Fahrenheit (15-25°C). Winters are mild, with highs in the 50s and 60s Fahrenheit (10-20"}],"stop_reason":"max_tokens"}
A failed curl with function calling
$ curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d '{
"anthropic_version": "vertex-2023-10-16",
"max_tokens": 100,
"tools": [{
"name": "get_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The unit of temperature, either \"celsius\" or \"fahrenheit\""
}
},
"required": ["location"]
}
}],"messages": [{"role": "user", "content": "What is the weather like in San Francisco?"}]
}' \
"https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models/${MODEL}:rawPredict"
> {"type":"error","error":{"type":"invalid_request_error","message":"tools: Extra inputs are not permitted"}}
from langchain-google.
After reading langchain-core
, I understand the other chat models using PydanticOutputParser
and JsonOutputParser
assume the LLM API can control output format as JSON like the JSON mode in OpenAI.
Until Claude on Vertex AI supports tools, we may be unable to support with_structured_output
in ChatAnthropicVertex
. Instead, we can use StructuredOutputParser
with giving a prompt by ourselves 1.
Please let me know if you have a different implementation idea. Thanks!
from langchain-google.
Related Issues (20)
- Additional kwargs key candidates_token_count already exists in left dict and value has unsupported type <class 'int'>. HOT 3
- index out out range exception ChatVertexAI - fix suggested in the ticket HOT 1
- VertexAISearchRetriever return Japanese documents as Unicode
- Multimodal FC does not work due to langchain incorrectly parsing as string HOT 1
- Slow initialization and invocation of ChatVertexAI with Gemini 1.5 Flash, while VertexAI performs faster HOT 2
- Error in DocumentAI: cannot import IndividualPageSelector
- ChatGoogleGenerativeAI function calling fails with enums in schema
- Model name cannot pass to GemmaLocalHF from langchain_google_vertexai HOT 1
- VertexAICheckGroundingWrapper breaks streaming HOT 4
- system prompt is not working when using ChatGoogleGenerativeAI
- GoogleDriveLoader hardcodes the Scope to a restricted scope HOT 2
- ChatGoogleGenerativeAI invoke docs are missing additional keyword params HOT 1
- MessageToDict() got an unexpected keyword argument 'always_print_fields_with_no_presence' HOT 2
- BigQueryVectorStore error when retrieving docs with filters (PR proposal)
- AttributeError: 'NoneType' object has no attribute 'prompt_feedback' HOT 1
- JSON Deserialization Error When Trying To Run Gemma Through Vertex AI HOT 5
- Discussion: How to implement complex filters for BigQueryVectorStore HOT 2
- Manual assignment of Document IDs in VertexaIVectorStore ignored HOT 1
- ChatGoogleGenerativeAI streaming stopping/freezing before completion HOT 4
- ChatGoogleGenerativeAI not return input/output token count HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from langchain-google.