jaqujaqu / cbmi_xai Goto Github PK

• Faithfulness: Ability to accurately represent the depiction of the underlying reasoning process behind the prediction of the model [2].

• Plausibility: describes the capability to offer explanations that are convincing to humans [2].
• Understandability: describes the ability to demonstrate the link between the input and the output of the model in relation to the system's parameters. It is often defined as the user’s cognitive perception of the model and its underlying functionality, the reasoning behind a model’s prediction, or the ability to understand why a model failed in a specific task [1], [3], [4].
• Trustfulness/Trustworthiness: is a factor directly influenced by the user's interaction with the system over time and through experience. It impacts the user’s comfort level when using the system. The user’s perception is directly connected to their confidence in the system's output [3], [4].
• Sufficiency: refers to providing enough information to the end user to establish causality [5].
• Usefulness/Helpfulness: usefulness is defined as the ability to provide explanations that make the model’s decisions more comprehensible to the user [6], [3].
• Satisfaction: pertains to how well users perceive their understanding of the explained system [6].

[1] G. Attanasio, D. Nozza, E. Pastor, and D. Hovy, ‘Benchmarking Post-Hoc Interpretability Approaches for Transformer-based Misogyny Detection’, in Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP, Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 100–112. doi: 10.18653/v1/2022.nlppower-1.11.
[2] A. Jacovi and Y. Goldberg, ‘Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?’, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online: Association for Computational Linguistics, 2020, pp. 4198–4205. doi: 10.18653/v1/2020.acl-main.386.
[3] P. Lopes, E. Silva, C. Braga, T. Oliveira, and L. Rosado, ‘XAI Systems Evaluation: A Review of Human and Computer-Centred Methods’, Appl. Sci., vol. 12, no. 19, p. 9423, Sep. 2022, doi: 10.3390/app12199423.
[4] S. Mohseni, N. Zarei, and E. D. Ragan, ‘A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems’, ACM Trans. Interact. Intell. Syst., vol. 11, no. 3–4, pp. 1–45, Dec. 2021, doi: 10.1145/3387166.
[5] E. Cambria, L. Malandri, F. Mercorio, M. Mezzanzanica, and N. Nobani, ‘A survey on XAI and natural language explanations’, Inf. Process. Manag., vol. 60, no. 1, p. 103111, Jan. 2023, doi: 10.1016/j.ipm.2022.103111.
[6] R. R. Hoffman, S. T. Mueller, G. Klein, and J. Litman, ‘Metrics for Explainable AI: Challenges and Prospects’. arXiv, Feb. 01, 2019. Accessed: Jul. 16, 2023. [Online]. Available: http://arxiv.org/abs/1812.04608