Comments (5)
I suppose that the embeddings were not calculated correctly, and consequently, you get a size mismatch with the class target file.
This is probably related to #9 , i.e. the fastBPE command was not in the path.
Please pull the latest source and try again (removing spurious files)
from laser.
@hoschwenk I have the latest source and the modification mentioned in #9 is also done. But the error is still there.
the link provided to download the XNLI 1.0 and XNLI-MT 1.0 seems to be broken. can this be the issue?
from laser.
I tied this solution, it solved a part of the problem, but this one still remain :
Training the classifier (see embed/xnli.log)
Traceback (most recent call last):
File "/home/trotignon/projects/LASER/source/nli.py", line 228, in
dim=args.dim, bsize=args.bsize, shuffle=True)
File "/home/trotignon/projects/LASER/source/nli.py", line 57, in LoadDataNLI
D = data_utils.TensorDataset(torch.from_numpy(nli), torch.from_numpy(lbl))
File "/home/trotignon/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataset.py", line 36, in init
assert all(tensors[0].size(0) == tensor.size(0) for tensor in tensors)
AssertionError
Even with the last version,
Thank you
from laser.
Indeed, it looks like that the XNLI link is broken or was moved elsewhere.
Sadly, I can't help you with this. If there is no data, the tool won't work :-)
Please have a look at the size of the created files - I guess that there is an error somewhere.
from laser.
Thank you very much, will look at it!
from laser.
Related Issues (20)
- Containerized Application, Corrupted file for bilstm.93langs.2018-12-26.pt HOT 2
- Change default tokenizer for sentence embeddings HOT 1
- The included Dockerfile and associated Flask app is out of date in multiple ways. HOT 2
- LASER3 encoders for English? (and other high res langs) HOT 2
- SPM models not available HOT 2
- Downloading models and tokenizers does not work on Windows HOT 1
- An error initializing English pipeline (on the MLH-dev branch)
- Suggestion: allow turning off punctuation normalization
- Suggestion: add normalization option in the
- Try decreasing the minimal versions of numpy and torch required by laser-encoders
- Reorganize the lists of languages
- Publish the performance of LASER2 and LASER3 encoders on each of the FLORES-200 languages
- Create a tutorial showing how LASER can be applied for downstream practical tasks (e.g. multilingual sentiment analysis) HOT 1
- Support for saving embeddings bin file in laser_encoder HOT 4
- Add 2-letter codes to the `laser_encoders` language list
- do it support qwen model? HOT 1
- How can I parallelize LASER embeddings? HOT 3
- Can't download: 403 error on some CC segments. HOT 1
- Unable to import LaserEncoderPipeline HOT 10
- Similarity calculation takes too long with LASER3 model, compared to XLM-R/LaBSE HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from laser.