Comments (13)
I'm unable to reproduce. Sincere apologies for the noise and wasting your time, and thanks for the model
from replitlm.
Can you run pip install --upgrade transformers
, and try again?
from replitlm.
From the error message, it looks like the CUDA drivers are not installed.
Can you test if you can run successfully the commands nvidia-smi
and nvcc --version
?
Also, we already list flash_attn
among the suggested dependencies -- check the README!
We haven't tested the model on M1/M2 Macs yet, so in case of further blockers, I can recommend to run with the default attention implementation in PyTorch.
from replitlm.
First of all, you need to install the latest versions of the following dependencies:
einops
sentencepiece
torch
transformers
is the section I read? If flash_attn
is listed, I don't see it
from replitlm.
(an m1 mac has no nvidia card so I don't think I can install nvcc? Too bad, but I get that some stuff can't run without an nvidia card)
from replitlm.
now I see that you listed it in the model description, but it appears to be necessary for inference as well, so it should be included in that list of required python packages is what I mean
from replitlm.
You don't need flash attention for inference -- it's a "nice to have" that makes inference faster, but to my knowledge it works only on NVIDIA GPUs (as you need CUDA).
In your case, you should load the model as indicated in the first half of that section:
from transformers import AutoModelForCausalLM
# load model
model = AutoModelForCausalLM.from_pretrained('replit/replit-code-v1-3b', trust_remote_code=True)
Hope this helps. Also, make sure to run on the latest version of the Transformers library!
from replitlm.
That's exactly what I did that caused the error to occur!
from replitlm.
I will do so tomorrow (I have to re-download the model now), but I was working in a clean virtualenv
from replitlm.
(which I assume means pip will download the newest version of a lib? But maybe that assumption is false if there's a previously cached version?)
from replitlm.
No problem! Glad it worked in the end :)
from replitlm.
I have mac m2 max 32 GB. pip install --upgrade transformers
has worked perfectly for me thanks @pirroh
from replitlm.
for me it doesn't work with pip install --upgrade transformers
from replitlm.
Related Issues (20)
- Sync
- Replit
- Challenge 1.1
- Challenge 1.1
- Challenge 1.2
- Clothes
- GPU hosted demo link broken in README
- HTML, CSS, JS
- Maachudo
- Model fails with AttributeError: 'ReplitLMTokenizer' object has no attribute 'sp_model' HOT 1
- Login page
- .html
- HTML, CSS, JS (9)
- HTML, CSS, JS (9)
- HTML, CSS, JS (9)
- Web minta thr @wahyu
- HTML, CSS, JS (10)
- OpenAi_Image_Generator
- Guess HOT 2
- switch doesn't work in cpp with some cases
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from replitlm.