Comments (4)
The last update broke this, but you can fix this in tokenization.py, you have to add this after vocab_file = pretrained_model_name
:
if os.path.isdir(vocab_file):
vocab_file = os.path.join(vocab_file, "vocab.txt")
from transformers.
Yes :-) There is a new release planned for tonight that will fix this (among other things, basically all the other open issues).
from transformers.
Ok, this is now included in the new release 0.3.0 (by #73).
from transformers.
Thank you, is it fair to assume that this will get accepted as an issue and fixed in a future update/release?
from transformers.
Related Issues (20)
- Bad request: Can't load config for 'None'. Make sure that: - 'None' is a correct model identifier listed on 'https://huggingface.co/models' - or 'None' is the correct path to a directory containing a config.json file HOT 3
- Trainer with resume_from_checkpoint does not work with multiple Peft Adapters HOT 1
- `generate` method does not work with `use_cache=False` in `Llama-2` model with `model.config._attn_implementation = "eager"` HOT 3
- add `stream` to pipeline parameters HOT 5
- Trainer/accelerate doesn't save model when using FSDP with SHARDED_STATE_DICT
- In attempting to install Apple's new OpenELM, huggingface's transformers appears to want us to ask for permission from FaceBook/Meta? HOT 6
- What should I do if I want to get a past dev version like v4.9.0.dev0 HOT 4
- Why is DeformableDetrForObjectDetection slower with bfloat16 than float32? HOT 8
- RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn HOT 5
- listing train_dataloader sampler throws out of memory error HOT 1
- Observing weird downwards jump in loss after checkpoint reloading in a DDP setting HOT 2
- GPTNeoX with use_cache=False uses significantly more memory than use_cache=True
- Removing model layers throws an index error. HOT 1
- cannot load model back due to [does not appear to have a file named config.json] HOT 5
- Inference bug of the MoE GPTQ models HOT 1
- Crash when trying to import pipeline when using TPUv2 in GoogleColab HOT 2
- Question about iterable inputs for Pipeline HOT 1
- [Question]: Can I obtain the probability from generated text? HOT 2
- [BUG] DataCollatorForSeq2Seq with PaddingStrategy.MAX_LENGTH may not pad labels HOT 3
- KeyError: 'shortest_edge' when loading Kosmos-2 model from local files HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transformers.