Comments (1)
I ran into the same problem while running the sample cases.
Traceback (most recent call last):
File "ms_marco_eval.py", line 20, in <module>
from rouge.rouge import Rouge
File "/Users/justincho/Desktop/Imago/Computer comprehension/MSMARCOV2/Evaluation/rouge/rouge.py", line 86
imgIds = list(gts.keys())
If you happened to get this error, just go to the rouge.py file and add another ")" to the end of line 86
I then got the following error:
OSError: [E050] Can't find model 'en_core_web_lg'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
This means you don't have this particular model from spacy downloaded, even if you have installed spacy. Run the following in your terminal:
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.0.0/en_core_web_lg-2.0.0.tar.gz
I actually still have a problem with one set of the sample files when I run
. run.sh sample_test_data/no_answer_test_references.json sample_test_data/no_answer_test_candidates.json
. run.sh sample_test_data/dev_as_references.json sample_test_data/dev_first_sentence_as_candidates.json
which state that AssertionError: Reference and candidate files must share same query ids
I believe this error might be intended for the no_answer set, but I think it shouldn't return this error for the dev_as_references set.
The script will work fine for the remaining sets:
. run.sh sample_test_data/sample_references.json sample_test_data/sample_candidates.json`
. run.sh sample_test_data/same_answer_test_references.json sample_test_data/same_answer_test_candidates.json
I hope this helped.
from msmarco.
Related Issues (20)
- MSMARCOV2/Ranking/README.md is not formatted correctly HOT 1
- How were passage reranking triples generated? HOT 2
- keyerror HOT 3
- Cannot find the top1000.eval for testing HOT 1
- Full Document May be incorrect tokenization in document_text HOT 2
- Collection paragraph metadata HOT 3
- BM25 relevance values for top 1000 eval/dev? HOT 1
- Training data with QID and PID HOT 2
- Broken eval script link in Ranking/README.md file HOT 1
- encoding, top1000.train, qrels.train HOT 1
- [encoding,Â] top1000.dev.tsv HOT 6
- Need more explanation about Reranking Dataset HOT 1
- KeyError for converttowellformed.py HOT 1
- Different number of queries in collectionandqueries.tar.gz and top1000.dev.tar.gz HOT 1
- Invalid line breaks in the top1000 TSV files of the reranking datasets HOT 4
- Partially duplicated passages extracted HOT 4
- Passage IDs in Qna Dataset HOT 1
- OpenKPAnnotations.tsv for Key Phrase Extraction HOT 1
- dev_as_references.json is from V1? HOT 2
- Regarding the Test Set for Q&A HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from msmarco.