An unofficial rust-based client library to interact with the Google Gemini generative AI API
Home Page: https://docs.rs/google-generative-ai-rs/
License: MIT License
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Describe the bug
stream iter response will block until SSE stream ends
To Reproduce
Steps to reproduce the behavior:
use futures::stream::{self, StreamExt};
let response = client.post(30, &txt_request).await?;
if let Some(gemini) = response.streamed() {
stream::iter(&gemini.streamed_candidates)
.for_each(|r| async {
output_response(r).await;
})
.await;
}Expected behavior
output_response one by one
Screenshots
If applicable, add screenshots to help explain your problem.
Version:
Additional context
Add any other context about the problem here.
Describe the bug
The public Gemini API supports streamed API responses. See: Stream Generate Content in API docs.
When using a 'streamGenerateContent' generates a '403' error on Public Gemini API.
Error: GoogleAPIError { message: "HTTP Error: 403: Forbidden", code: Some(403) }To Reproduce
In examples/test_request.rs
// Either run as a standard text request or a stream generate content request
// let client = Client::new(env::var("API_KEY").unwrap().to_string());
// UNCOMMENT THIS SECTION
let client = Client::new_from_model_reponse_type(
google_generative_ai_rs::v1::gemini::Model::GeminiPro,
env::var("API_KEY").unwrap().to_string(),
google_generative_ai_rs::v1::gemini::ResponseType::StreamGenerateContent,
);API_KEY=[YOUR_API_KEY] RUST_LOG=info cargo run --package google-generative-ai-rs --example text_request
Compiling google-generative-ai-rs v0.1.5 (/Users/[USER_NAME]/repos/google-generative-ai-rs)
Finished dev [unoptimized + debuginfo] target(s) in 0.91s
Running `target/debug/examples/text_request`
Error: GoogleAPIError { message: "HTTP Error: 403: Forbidden", code: Some(403) }Expected behavior
It should support 'streamGenerateContent' as expected, as per the Vertex AI endpoint and as per cURL
Version (please complete the following information):
Additional context
Using cURL works fine for the Public API endpoint.
curl https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:streamGenerateContent?key=${API_KEY} \
-H 'Content-Type: application/json' \
--no-buffer \
-d '{ "contents":[{"parts":[{"text": "Write long a story about a magic backpack."}]}]}' As per Gemini docs the current client is not feature complete and is missing function calls.
We need to incorporate as much of the Gemini documentation into easy-to-follow documentation on this repository. In particular, there is little to no narrative around the examples.
good lib just testing it out but noticed some of the types are identical but not combined i.e. Content in response and request.
if you want i can probably get a fix at some point but really busy for the next little bit
Okay, show time! To be honest this has been a missing feature that has meant that I have not spent much time on this recently as it is vital for any meaningful generative usage that isn't plain chat.
Google added the feature to the API for both public and Vertex endpoints earlier in the week.
I will try to look at adding this in during the next week or so.
The new Gemini models (1.0) have been released on 02/15/2024, these include gemini-1.0-pro-latest
(using get_model)
ModelInformationList {
models: [
ModelInformation {
name: "models/gemini-1.0-pro",
version: "001",
display_name: "Gemini 1.0 Pro",
description: "The best model for scaling across a wide range of tasks",
input_token_limit: 30720,
output_token_limit: 2048,
supported_generation_methods: [
"generateContent",
"countTokens",
],
temperature: Some(
0.9,
),
top_p: Some(
1.0,
),
top_k: Some(
1,
),
},
ModelInformation {
name: "models/gemini-1.0-pro-001",
version: "001",
display_name: "Gemini 1.0 Pro 001",
description: "The best model for scaling across a wide range of tasks. This is a stable model.",
input_token_limit: 30720,
output_token_limit: 2048,
supported_generation_methods: [
"generateContent",
"countTokens",
],
temperature: Some(
0.9,
),
top_p: Some(
1.0,
),
top_k: Some(
1,
),
},
ModelInformation {
name: "models/gemini-1.0-pro-latest",
version: "001",
display_name: "Gemini 1.0 Pro Latest",
description: "The best model for scaling across a wide range of tasks. This is the latest model.",
input_token_limit: 30720,
output_token_limit: 2048,
supported_generation_methods: [
"generateContent",
"countTokens",
],
temperature: Some(
0.9,
),
top_p: Some(
1.0,
),
top_k: Some(
1,
),
},
ModelInformation {
name: "models/gemini-1.0-pro-vision-latest",
version: "001",
display_name: "Gemini 1.0 Pro Vision",
description: "The best image understanding model to handle a broad range of applications",
input_token_limit: 12288,
output_token_limit: 4096,
supported_generation_methods: [
"generateContent",
"countTokens",
],
temperature: Some(
0.4,
),
top_p: Some(
1.0,
),
top_k: Some(
32,
),
},
ModelInformation {
name: "models/gemini-pro",
version: "001",
display_name: "Gemini 1.0 Pro",
description: "The best model for scaling across a wide range of tasks",
input_token_limit: 30720,
output_token_limit: 2048,
supported_generation_methods: [
"generateContent",
"countTokens",
],
temperature: Some(
0.9,
),
top_p: Some(
1.0,
),
top_k: Some(
1,
),
},
ModelInformation {
name: "models/gemini-pro-vision",
version: "001",
display_name: "Gemini 1.0 Pro Vision",
description: "The best image understanding model to handle a broad range of applications",
input_token_limit: 12288,
output_token_limit: 4096,
supported_generation_methods: [
"generateContent",
"countTokens",
],
temperature: Some(
0.4,
),
top_p: Some(
1.0,
),
top_k: Some(
32,
),
},
ModelInformation {
name: "models/embedding-001",
version: "001",
display_name: "Embedding 001",
description: "Obtain a distributed representation of a text.",
input_token_limit: 2048,
output_token_limit: 1,
supported_generation_methods: [
"embedContent",
"countTextTokens",
],
temperature: None,
top_p: None,
top_k: None,
},
],
}
As per the Gemini documentation the current client is missing embedding.
The original code was written quickly straight after the Gemini API was released. There are poor parts of the code:
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.