Tokenizer version is in conflict with transformers version about ai-for-beginners HOT 5 OPEN

Fixing tokenizer version in .devcontainer/requirements.txt should fix the issue

from ai-for-beginners.

@GanGral I reproduced the issue, I will commit the change once I confirm it fixed the issue and will create the Pull Request on this one, thank you!

from ai-for-beginners.

Hey @omar908, thanks for coming back this quickly. Yes, that make sense a lot, so unfortunately we will have to wait for someone to approve that PR.

I am planning to check out this course, in following week or two, so if I encounter any other issue (already did), I will try to summarise all of them, in one Issue post.

from ai-for-beginners.

@omar908 This issues is still a thing. And while trying to for example,
pip install -r ./.devcontainer/requirements.txt
following message will pop up

The conflict is caused by:
    The user requested tokenizers==0.10.3
    transformers 4.30.0 depends on tokenizers!=0.11.3, <0.14 and >=0.11.1

Something like this in requirements.txt file, would resolve the issue tokenizers>=0.11.1,!=0.11.3,<0.14 or like in the suggested PR tokenizers==0.13.3. Can you check on the above linked PR?

from ai-for-beginners.

Hey @l3ka , the PR above is the one I created a while back, but seems it is still awaiting approval from someone of this project. Won't allow me to merge it until then, not much I can do on my end, sadly.

from ai-for-beginners.