Fairpy is an open source Toolkit for measuring and mitigating biases in large pretrained language models. It currently supports a wide range of bias detection tools and Bias Mitigation techniques along with interfaces for augmenting corpus, plugging in custom language models and extending the package to include new techniques.
Paper Link https://arxiv.org/abs/2302.05508
- Bias Detection Methods
- Top K Overlap
- Hellinger Distance
- F1 Score
- Honest
- Log Probability
- StereoSet
- WEAT/SEAT Score
- Bias Mitigation Methods
- DiffPruning
- Entropy Attention Regularization
- Counter Factual Data Augmentation
- Null Space Projection
- Self Debias (Incomplete)
- Dropout Regularization
- Models
- CTRL
- GPT-2
- OpenAI-GPT
- TransfoXL
- BERT
- DistilBERT
- RoBERTa
- AlBERT
- XLM
- XLNet
from fairpy import BiasDetection.BiasDetectionMetrics as BiasDetectionMetrics
maskedObj = BiasDetectionMetrics.MaskedLMBiasDetection(model_class = 'bert-base-uncased')
maskedObj.WeatScore(bias_type='health')from fairpy import BiasDetection.BiasDetectionMetrics as BiasDetectionMetrics
causalObj = BiasDetectionMetrics.CausalLMBiasDetection(model_class = 'gpt2')
causalObj.stereoSetScore(bias_type='gender')from fairpy import BiasMitigation.BiasMitigationMethods as BiasMitigationMethods
MaskedMitObj = BiasMitigationMethods.MaskedLMBiasMitigation(model_class='bert-base-uncased')
model, tokenizer = MaskedMitObj.NullSpaceProjection('bert-base-uncased', 'BertForMaskedLM', 'race', train_data='yelp_sm')from fairpy import BiasMitigation.BiasMitigationMethods as BiasMitigationMethods
CausalMitObj = BiasMitigationMethods.CausalLMBiasMitigation(model_class='gpt2')
model, tokenizer = CausalMitObj.DropOutDebias('gpt2', 'religion', train_data='yelp_sm')
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent