Comments (2)
You can wrap all the perturbation functions into one. Here is an example of how to do that if you just want to run each of the perturbation functions in the original input (you can remove all of the meta
checking if you are not using that argument):
import checklist
from checklist.perturb import Perturb
import spacy
nlp = spacy.load('en_core_web_sm')
def wrap_multiple_perturbs(perturb_fns):
def rfn(example, *args, **kwargs):
use_meta = kwargs.get('meta', False)
ret = []
meta = []
for fn in perturb_fns:
r = fn(example, *args, **kwargs)
if not r:
continue
meta = []
if use_meta:
r, meta = r
ret.extend(r)
meta.extend(meta)
if use_meta:
ret = (ret, meta)
return ret
return rfn
fn = wrap_multiple_perturbs([Perturb.change_names, Perturb.change_location])
fn(nlp('John is a man who lives in Turkey.'), n=3)
['Dylan is a man who lives in Turkey.',
'Connor is a man who lives in Turkey.',
'Nathan is a man who lives in Turkey.',
'John is a man who lives in Egypt.',
'John is a man who lives in Mozambique.',
'John is a man who lives in Japan.']
Note that this is running each function independently. It's a little more complicated if you want to run them in sequence, because the order matters and the output of these functions are strings while the inputs are spacy.doc
. Anyway, here is a version you can tweak to your needs (removed meta just for conciseness):
def wrap_perturbs_in_sequence(perturb_fns):
def rfn(example, *args, **kwargs):
ret = []
examples_to_run = [example]
for fn in perturb_fns:
new = []
for example in examples_to_run:
r = fn(example, *args, **kwargs)
if not r:
continue
new.extend([nlp(x) for x in r])
examples_to_run.extend(new)
return [x.text for x in examples_to_run if x.text != example]
return rfn
fn = wrap_perturbs_in_sequence([Perturb.change_names, Perturb.change_location])
fn(nlp('John is a man who lives in Turkey.'), n=3)
[John is a man who lives in Turkey.,
Austin is a man who lives in Turkey.,
Patrick is a man who lives in Turkey.,
John is a man who lives in Angola.,
John is a man who lives in Nigeria.,
John is a man who lives in Germany.,
Austin is a man who lives in Mozambique.,
Austin is a man who lives in Vietnam.,
Austin is a man who lives in Colombia.,
Patrick is a man who lives in Ghana.,
Patrick is a man who lives in Bangladesh.,
Patrick is a man who lives in United Kingdom.,
Robert is a man who lives in Vietnam.,
Robert is a man who lives in India.,
Robert is a man who lives in Ghana.]
from checklist.
Thanks a lot for great implementations !
from checklist.
Related Issues (20)
- IndexError when running test HOT 3
- Release of fine-tuned checkpionts HOT 1
- mltests module? HOT 1
- Uhandled TypeError when running Sentiment analysis test suite HOT 6
- Encoding issue in Qindows with TestSuite.to_raw_file HOT 1
- Text Generation model for negation HOT 1
- 4. The CheckList process.ipynb : pipe([example]) incompatible type HOT 4
- How to load in tests and templates from tsv HOT 1
- Potential Case-Sensitivity Issue When Testing SQuAD HOT 1
- Using functions to convert slot words into different forms HOT 1
- Invoking pip in setup.py breaks custom flags HOT 5
- Is there a way to save all the failed test cases in some text file or in a list? HOT 1
- Most test cases of reducing test_type discarded HOT 1
- Scale checklist to classical ML models HOT 1
- Any suggestion for checklist for multi-class Intent Prediction models for Chatbots? HOT 1
- Any suggestion on how to block masked words? HOT 1
- IndexError: tensors used as indices must be long, byte or bool tensors HOT 2
- Requesting hyperparameter details HOT 1
- editor.synonyms breaks on specific input HOT 1
- Minor fix to Wordnet text generation score HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from checklist.