AI-Assisted Decision-Making Framework

Asked unconditionally, do explanations help, this literature answers no. Asked conditionally, specifying model performance, task difficulty, risk and explanation form, it answers yes under stated conditions and no otherwise.

• On the negative side, Alufaisan and colleagues ran the three-arm design with a no-AI control, an AI-only arm and an AI-plus-explanation arm, and found the AI itself carried the benefit while explanations added nothing.
• De Brito Duarte and colleagues found the effect of explanation on trust jointly conditional on system performance and risk level, and only feature importance moved anything, with counterfactuals indistinguishable from no explanation for lay users.
• On the positive side, Vasconcelos and colleagues obtain clear benefits in hard tasks and with salient explanations, and Leichtmann and colleagues find explanations improving accuracy on precisely the items where the classifier erred, while lowering trust in a way they read as better calibration.

Knowing that a model is 80% accurate tells the decision-maker nothing about the case in front of them. What supports selective reliance is a usable model of the error boundary, meaning where in the input space the system fails, and the properties of that boundary matter as much as the accuracy figure attached to it.

The two Bansal 2019 papers establish both halves. Mental model quality mediates the relationship between AI accuracy and team performance, and boundaries that are less parsimonious, higher-dimensional or stochastic are learned poorly or not at all: a two-sided stochastic boundary was never learned, with the proportion of optimal choices flat near 50% across rounds. The corollary is that an update improving accuracy can reduce team performance by invalidating the boundary the human has already learned.

Most interventions that make people less willing to accept the AI make them less willing to accept it when it is right as well as when it is wrong. The two movements cancel, which is why so many designs succeed on one reliance measure and vanish on decision quality. A reported reduction in over-reliance is therefore uninformative unless the corresponding under-reliance figure is given alongside it.

What determines whether an explanation improves a decision is not how much it explains but whether the user can hold it against the evidence and see a mismatch. Explanations that are inspectable at a glance, such as similar cases shown as images or a highlight drawn on the artefact itself, let people form their own reading first and make the model's errors visible. Explanations that must be reasoned about in the abstract, such as feature weights, counterfactuals or fluent prose, are rhetorically dominant, suppress the user's own reading of the case, and are accepted without being checked.

Fluency makes this worse: the most conversational interface in the base produced the worst self-reliance. The mechanism is distinct from explanation quality, since a faithful explanation in an uncheckable form still displaces judgement.

Accepting the AI is a strategy with a cost of nearly zero, and verifying it is a strategy with a cost set by the task, the time available and the legibility of the explanation. People choose between them rationally, so over-reliance rises whenever verification becomes expensive relative to the penalty for error, and falls when an intervention makes verification cheap or makes error costly.

An explanation that looks sound is taken as evidence that the recommendation is sound, and the check that would have caught the error is never performed. The harm arrives through the form of the explanation looking right rather than through the effort it demands, which distinguishes this mechanism from the engagement-cost one: here the explanation is cheap to read and that is precisely the problem.

• Cabitza and colleagues 2024 give the sharpest demonstration and name it the XAI halo effect. Participants solving logic puzzles with verifiable answers received correct advice accompanied by either a relevant or an irrelevant explanation, and did significantly worse with the irrelevant one, a large effect. They could have checked the advice against the puzzle directly and did not.

There is no stable law relating expertise to reliance in this evidence base, and claims of the form novices over-rely while experts resist are not supported. The same class of support raises novices to expert level in one clinical study and reaches only the experienced in another; experts are more discriminating in some settings and measurably less so in others. Measured knowledge frequently predicts nothing at all.

Reflective designs encourage the decision-maker to think critically before acepting or rejecting an AI advice. This can be done by withholding the recommendation, or adding a layer that argues with it. In every study in the base that measured over-reliance under such a design, over-reliance fell. The effect is more consistent than the effect of explanations.

When a recommendation is present the decision-maker can reach an answer without processing the explanation, and does. Remove the recommendation while keeping the same explanatory content and the explanation becomes the only route to an answer, so it must be processed, and processing it leaves knowledge behind. The claim is about skill acquisition specifically: performance and learning are separable outcomes that interface design can trade against each other.

• Gajos and Mamykina isolate this holding explanation content identical across three experiments and finding learning only in the condition where the conclusion was withheld. The update protocol, which reduces over-reliance, produced no learning at all.

Deliberation only protects a group if there is something to deliberate about. A bare recommendation gives two or three people nothing to discuss over, so the shortest route to the consensus the task demands is to adopt the AI’s answer. Give the group material: an explanation, a confidence figure, an agent that argues back, and the discussion lengthens, and it is the discussion that tracks the drop in over-reliance.

Removing the AI recommendationfrom the AI advice removes the shortcut. Whoever cannot perform the interpretation the recommendation used to spare them is left worse off than they would have been with a directive system, so the benefit of reflective design is conditional on the user's capacity to supply what the system withholds.

• Cabitza and colleagues 2025 give the strongest and most uncomfortable version. Judicial AI, which issues no recommendation and instead presents one activation map supporting the presence of a fracture and one supporting its absence, produced a large significant benefit for more experienced clinicians. For less experienced clinicians the point estimate was negative. The authors name this as the study's main caution in their own terms: a protocol that demands interpretation may not be safe for those least equipped to interpret.
• Li and colleagues report that both moderators of their reflection effect were individual capacities rather than states, so who benefited was largely determined before the intervention began. Tian and colleagues found no learning effect for any treatment and no improvement on minority-label cases for any form of assistance, so the reflective arms did not lift the harder cases for a general population.

Designs that protect people from the AI's errors by making them do more cognitive work are experienced as worse systems. It applies to interventions that work by raising the cost of accepting the AI, such as cognitive forcing functions or forced deliberation.

• It does not apply to interventions that reduce over-reliance by making verification cheaper, which people willingly pay for: participants in Vasconcelos and colleagues assigned the highest subjective utility to precisely the explanation formats that produced the largest reductions in over-reliance.

Human-first protocols are widely recommended on the grounds that committing to a judgement before seeing the AI protects against anchoring. This mechanism records the counter-evidence: committing first can entrench a wrong prior through conservatism bias, and seeing the AI as a challenge to a position already taken can trigger rejection of correct advice.

Nothing about the information changes, only the register in which it is asserted, and reliance changes anyway. The mechanism is social and communicative rather than cognitive, and the optimal register appears to depend on the recipient rather than being a fixed property of good design.

• An imposing tone and a suggestive tone delivering identical breast-cancer diagnostic content produced different diagnoses, with novices more accurate under the assertive register and experts slightly better under the suggestive one. The gain came almost entirely from novices ceasing to overturn correct recommendations, with acceptance of incorrect ones essentially unchanged, at no measurable workload cost and with diagnosis time cut by a quarter. (Calisto et. al.)
• Li and colleagues supply the strongest control. Four chatbots drew on an identical knowledge base and differed only in social register, and the register that changed behaviour was opinion agreement, which significantly reduced the likelihood of revising a decision under contradicting evidence. Direct praise and self-deprecation, which participants recognised as flattery, moved perceptions and confidence but not decisions. The manipulation that worked was the one nobody detected.
• De Brito Duarte and colleagues extend the claim from register to modality: the same TreeSHAP or LORE explanation delivered through a chatbot rather than as static text significantly raised agreement with the recommendation and lowered the rate of declining to commit, with no gain in accuracy, which the authors read as a persuasive rather than an epistemic effect.
• Munyaka and colleagues supply a fourth variant, source framing: with the partner's advice hard-coded and identical, labelling it human rather than AI raised adoption from 1.44 to 2.10 of five suggestions.

Lowering the price of checking AI advice does nothing when checking is not possible. Where the ground truth is not available at decision time, where the outcome is genuinely uncertain, or where the human simply lacks the knowledge the case requires, there is no basis on which to override, and reliance rises by default.

• Salimzadeh and colleagues manipulate uncertainty directly and find participants leaning harder on the AI precisely where they were least able to distinguish good advice from bad. Green and Chen produce the strongest version across six presentation conditions: participants could evaluate neither their own accuracy nor the model's in any of them, an empty cycle of oversight.
• Bucinca and colleagues 2025 supply the blunt statement of the consequence. Across every condition of their study, including the contrastive explanations that produced real learning gains, accuracy on the items the AI got wrong fell to 0.14 against 0.29 unaided. Assistance roughly halved unaided performance on exactly the cases where the model failed, and no explanation design in that study altered it.

This is the strongest argument in the base against assuming that human oversight, as required by EU AI Act Article 14 and comparable provisions, is protective in prognostic or high-uncertainty deployments. Oversight presupposes a capacity to verify that the task itself may not afford.

Stakes changes how selectively people take AI advice. Three studies raise the stakes in three different ways: a monetary bonus, an in-game consequence, and a change of domain. What changes when stakes are high is the response to worng advice of the advice that challenges the decision-maker, not the response to advice that confirms them.

• Over-reliance, defined as acceptance of incorrect AI predictions, was lower under the larger incentives (Vasconcelos et. al.)
• When AI advice contradicted initial decision-maker choice, adoption was significantly higher in the high-risk ETF task than in the low-risk dating task (beta = 0.843, p < 0.001). In the authors' words, task risk selectively amplifies the influence of contradictory, rather than confirmatory, AI advice. (Li et. al.)
• In mushroom classification task, when the stakes are higher, people reported lower levels of trust compared to when the stakes are lower, but only when the AI is underperforming (de Brito Duarte et. al.)

The value of a second AI opinion comes from its errors falling on different cases. Advisors drawn from the same system share an error boundary, so the first addition carries most of the available gain and further additions carry almost none. This means that adding independent AI advisors might improve performance, but the effect is not monotonic, 5 AI advisors do not improve decision performance compared to 3.

Complementarity is a property of the relationship between two error distributions. A system weaker than the average human produces a team better than either when its mistakes fall on different cases; a system stronger than the human produces no gain at all when their mistakes coincide. This explains a common finding where AI assistance lifts people above their unaided performance but never above the AI alone: the overlap sets the ceiling, and no interface change moves it. The practical implication is that model selection for deployment should be driven by error independence from the intended user, which is almost never what accuracy benchmarks measure.

Control over the information flow shapes the reliance profile. In the studies supporting this mechanism the content of the advice is held constant and only the user's authority over it changes: whether a second opinion arrives unbidden or on request, and whether the recommendation can be adjusted or only accepted or rejected. Both manipulations move trust, understanding and compliance while the underlying advice stays the same.

In group decision making, an independent decision mechanims where several individuals each decide with the AI and their judgements are combined by an agregation rule, is better than any of them making the decision alone or in deliberation. The reason is statistical: the cases on which a given person is talked out of a correct answer are not the same cases for the next person, so uncritical acceptance of a bad recommendation rarely survives a majority. Nothing about the individual's reliance improves; the errors are diluted.

Whether AI advice is taken depends less on its content than on whether it matches the judgement the person had already formed. Congruent advice is weighted more heavily and raises confidence; incongruent advice is discounted even when correct. The cases where the AI could correct a human error are the cases where its advice gets discounted, while the cases where human and machine are wrong together produce the strongest agreement and the highest confidence.

When several AI advisors speak, people respond to how many of them agree rather than to what that agreement is worth. In both supporting studies the agreement carries no evidential content by construction, and it moves behaviour anyway, which is what identifies the pressure as social rather than informational. The effect is not monotonic in consensus: a single dissenting voice relieves it, and wide disagreement replaces it with confusion.