- Lizhen Liang, Ph.D. student, Syracuse University
- Daniel E. Acuna, iSchool, Syracuse University
pip install git+https://github.com/sciosci/psycho_ai
Extract PSE and JND of word embedding models:
from psycho_ai import two_afc
embedding = two_afc.get_glove_100d()
target_occupations = ['accountant', 'supervisor', 'worker', 'clerk', 'instructor', 'inspector',
'electrician','appraiser', 'administrator', 'receptionist', 'advisor', 'chemist',
'planner','paralegal', 'veterinarian', 'psychologist', 'baker', 'teacher',
'lawyer','nutritionist', 'hairdresser','pathologist', 'surgeon', 'practitioner',
'carpenter']
female_male_pairs = [['woman', 'man'],
['female', 'male'],
['she', 'he'],
['her', 'him'],
['hers', 'his'],
['daughter', 'son'],
['girl', 'boy'],
['sister', 'brother']]
pse_score = two_afc.pse(embedding, target_occupations, female_male_pairs)
jnd_score = two_afc.jnd([embedding_A,embedding_B],target_occupations, female_male_pairs)
PSE Score
{'advisor': 0.23486328125,
'architect': 0.24462890625,
'officer': 0.29150390625,
...
...
...
'therapist': 0.7587890625,
'hairdresser': 0.76513671875,
'hygienist': 0.830078125}
JND Score
{'janitor': 0.0010945320129394534,
'technician': 0.0012115716934204103,
'administrator': 0.0013679265975952148,
...
...
...
'supervisor': 0.021092605590820313,
'architect': 0.022199630737304688,
'plumber': 0.02711360454559326}
two_afc.plot_pse(pse_score)
Please refer to
- Liang, L., & Acuna, D. E. (2020, January) . Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models . In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency ( pp. 403-412).