Comments (1)
marcotcr
commented on July 18, 2024
This test has the following description:
A model should not be more confident on "I used to think X" when compared to "X", e.g. "I used to love this airline" should have less confidence than "I love this airline
Unfortunately, this can only fail if there is a margin for the model to be 'more confident' with 'I used to think X', i.e. if f(X) is not very close to 100% confidence. Thus, this test doesn't really work for really confident models, which is probably your case here. There is no way to fix it : )
from checklist.
Related Issues (20)
- editor.synonyms breaks on specific input HOT 1
- Minor fix to Wordnet text generation score HOT 1
- IndexError when running test HOT 3
- Release of fine-tuned checkpionts HOT 1
- mltests module? HOT 1
- Uhandled TypeError when running Sentiment analysis test suite HOT 6
- Encoding issue in Qindows with TestSuite.to_raw_file HOT 1
- Text Generation model for negation HOT 1
- 4. The CheckList process.ipynb : pipe([example]) incompatible type HOT 4
- How to load in tests and templates from tsv HOT 1
- Potential Case-Sensitivity Issue When Testing SQuAD HOT 1
- Using functions to convert slot words into different forms HOT 1
- Invoking pip in setup.py breaks custom flags HOT 5
- Is there a way to save all the failed test cases in some text file or in a list? HOT 1
- Google colab HOT 1
- Need to pin notebook version - 7.0.1 breaks install - AttributeError: module 'notebook' has no attribute 'nbextensions' HOT 3
- suite.visual_summary_table() API results in JS error while loading the html table
- Read the 'pkl' file under Windows, prompt 'an integer is required (get type bytes)'
- Reproducibility Issues When Templating
- do you have some progress in this directions since 2 years ago
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from checklist.