arXiv preprint ·
XAI Evaluation Cards: A Practical Method for Designing Human-Centred XAI Evaluations
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Evaluating explainable AI (XAI) systems from a human-centred approach requires researchers to select from numerous evaluation dimensions and measures, often in an ad hoc and fragmented manner. This paper introduces a method to help HCI, computer science, designers and social science researchers systematically evaluate XAI systems. The approach is based on an updated XAI-specific evaluation framework derived from an analysis of 82 studies. Using this framework, we developed a card-sorting method with 36 cards to help researchers prioritise relevant evaluation aspects. The process was tested with two research groups (n = 13) across five projects. The XAI Evaluation Cards are available as a printable appendix, along with an online repository of methods from previous XAI studies. Although not exhaustive, our findings indicate that the card-sorting approach can organise and streamline the design of the evaluation process, encouraging a more comprehensive and multidisciplinary assessment of XAI systems in research and development.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Human-Computer Interaction
ABDA-NL: A Natural-Language Scenario Explorer for Argument-Based Reasoning
Asterism: Exploring and Synthesizing Scattered Observations into Literature-Grounded Hypotheses and Theories
Cybernetic and Epistemic: A Missing Vocabulary for Trustworthy Agentic Delegation
DataWeave: Deploying Human-LLM Analytics for Exploratory Structured Data Analysis
Scaling Peer Assessments: An Integrity Report from a Large Engineering Internship
Get the few papers worth your time in Human-Computer Interaction every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.