Comments (2)
@kathyyu-google can you please help or refer to someone? Thanks
from vertex-ai-samples.
Thank you @MangoHiller for bring this to our attention. We were able to reproduce your error with long prompts, which led to high memory usage. We found that the error could be avoided by decreasing the model server argument --gpu-memory-utilization
to 0.85. This argument is defined in the function deploy_model_vllm
and originally set to 0.9.
Marking this issue as closed. Please feel free to reopen if there are further comments or prediction errors. Thank you again!
from vertex-ai-samples.
Related Issues (20)
- Error using sample request code for openai server endpoint format HOT 1
- Update 3 "Open in..." links in notebook HOT 1
- Pytorch version sync with Cuda HOT 5
- Unable to connect to a repository HOT 6
- As always , code doesn't work HOT 2
- notebooks/community/bigquery_ml/bqml-online-prediction.ipynb fails with "InvalidArgument: 400 Error occurred in Explanation preprocessing. <class 'ValueError'> NodeDef mentions attr 'debug_name' not in Op<name=VarHandleOp; signature= -> resource:resource" HOT 2
- Using macros in Vertex AI Pipelines HOT 1
- Notebook https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/explainable_ai/sdk_custom_tabular_regression_online_explain_get_metadata.ipynb and others fail with internal 500 error when doing model.explain
- Export metrics in csv
- Issue with deploying via Hex-LLM, TPU serving solution built with XLA, which is being developed by Google Cloud. HOT 6
- "Using the `Trainer` with `PyTorch` requires `accelerate>=0.20.1`" when running "pytorch_text_sentiment_classification_custom_train_deploy.ipynb" HOT 4
- model_garden_pytorch_stable_diffusion_xl_1_0.ipynb notebook not working with SDXL-REFINER option. HOT 9
- model_garden_pytorch_stable_diffusion_xl_lora.ipynb not working with Civitai HOT 1
- Model Garden Gemma Deployment on Vertex - incomplete documentation about prediction response format HOT 1
- Gemma in Model Garden Deployment - confusing section on Chat Applications HOT 1
- Dataset creation
- Vertex AI pipeline - IndexError: Invalid key: 0 is out of bounds for size 0 HOT 17
- vertex ai pipeline key error root HOT 1
- Running LocalModel.build_cpr_model returns No such file or directory: 'docker'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vertex-ai-samples.