AI Trust: Can Explainable AI Enhance Warranted Trust?

A 3 by 2 by 2 mixed-design experiment on Prolific, with 215 participants recruited and 211 retained after attention checks.

Task. A purpose-built web game in which the player judges the edibility of mushrooms. Each screen showed the mushroom's characteristics, a cartoon depiction, and an AI recommendation to eat or avoid. Each game ran 12 mushrooms, with feedback after every decision on whether the choice was sound, and a final score counting edible and poisonous mushrooms consumed. The goal was to eat as many mushrooms as possible while avoiding poisonous ones. At the last stage of each game, participants stated whether they would delegate the upcoming action to the AI.

AI models. Both were decision trees trained on the UCI Mushroom dataset (8,124 entries, 22 categorical variables, binary edibility target). Feature selection reduced the input to eight features chosen for interpretability rather than pure predictive value: odour, cap colour, cap shape, cap surface, gill colour, gill size, gill spacing and ring number. Odour dominated the importance ranking; cap and gill colour were retained despite low importance because they align with lay mental models of mushrooms. The high-performance model was trained on 70% of the data with depth 12 and reached 96% test accuracy. The low-performance model was produced by flipping 30% of the edibility labels in the training data and retraining on the same features, reaching 60% test accuracy.

Factors. Three were manipulated. XAI model type, between-subjects, three levels: control with no explanation, LIME providing local feature importance, and DICE providing counterfactual explanations. Risk, between-subjects, two levels: in the high-risk condition eating one poisonous mushroom made the character sick and a second ended the game, while in the low-risk condition the character only fell ill after three poisonous mushrooms. AI performance, within-subjects, two levels: each participant played the game twice, once with each model, in counterbalanced order, keeping the same XAI type and risk level across both rounds.

Procedure. Consent, demographics, instructions and a tutorial, then the first game of 12 decisions, the delegation question, a questionnaire, then the second game at the other performance level and a second questionnaire.

Outcomes. Subjective trust was measured two ways: the multidimensional measure of trust (MDMT), averaging eight trait ratings (reliable, predictable, consistent, skilled, capable, competent, precise, transparent) on a 0 to 7 scale, and a single direct item asking about perceived trust in the same format. AI understandability used two items on how well the game helped the participant understand the system and how they rated their own understanding. XAI quality used four items on the sufficiency and satisfyingness of the information presented. Behavioural trust was the percentage of decisions on which the participant agreed with and acted on the AI's recommendation. Delegation was the binary choice to hand the final action to the AI.

Model performance dominated everything, and the effect of explanation depended on which trust measure was used.

Performance produced large main effects on every dependent variable, with by far the biggest effect sizes in the study: MDMT 5.34 against 4.05 (F(1,205) = 172.83, p < .001, partial eta squared = .46), direct subjective trust 4.84 against 3.46 (F = 125.64, p < .001, .38), XAI quality 4.85 against 4.11 (F = 92.03, p < .001, .31), AI understandability 4.53 against 3.92 (F = 45.34, p < .001, .18) and behavioural trust 0.87 against 0.80 (F = 37.71, p < .001, .16). Participants rated the explanations of the well-performing model as higher quality and rated their own understanding as better, even though the explanation technique was identical across the two models. The authors flag this as unexpected and offer two readings: an unambiguous high-performing model is genuinely easier to comprehend, or a well-performing model creates a misleading impression of understanding.

Only one explanation type worked, and only on self-report. LIME raised MDMT (F(2,205) = 3.50, p = .032, partial eta squared = .033; LIME 4.98 against DICE 4.60 and control 4.50), AI understandability (F = 4.10, p = .018, .038; LIME 4.70) and XAI quality (F = 8.61, p < .001, .078; LIME 5.01, DICE 4.40, control 4.04). Direct subjective trust showed the same trend but did not reach significance (F = 2.26, p = .10). Counterfactual explanations were statistically indistinguishable from no explanation at all on every measure, which the authors read as non-experts lacking the background to use counterfactual examples. H1 is therefore only partially supported: feature importance raised trust, counterfactuals did not.

The critical result for H2 is a dissociation between what people said and what they did. Self-reported trust showed no interaction between XAI type and performance on any measure: LIME participants reported elevated trust whether the model was 96% accurate or 60% accurate. Behavioural trust behaved oppositely, with a strong XAI by performance interaction (F(2,205) = 25.40, p < .001, partial eta squared = .20). Under high performance, LIME participants complied most (0.93 against 0.84 in DICE and control); under low performance, LIME participants complied least (0.75 against 0.81 DICE and 0.84 control). Feature importance explanations therefore made behaviour more discriminating while leaving stated trust undiscriminating. The authors' headline concern, that explanations can raise trust in an unreliable system, rests on the self-report side of this split; on the behavioural side the same explanations helped people back away from a bad model.

Risk mattered only when the model was weak. All four self-reported measures showed significant risk by performance interactions, with high risk depressing trust, understanding and perceived explanation quality relative to low risk only under low performance (MDMT F(2,205) = 10.02, p = .002; direct trust F = 5.99, p = .015; XAI quality F = 6.51, p = .011; AI understandability F = 4.33, p = .039). Under high performance no risk differences appeared at all. Behavioural trust was untouched by risk (F(1,205) = 0.566, p = .453). H3 is partially supported: risk lowers trust, but only where the model gives it reason to.