Human-Human-AI Triadic Programming: Uncovering the Role of AI Agent and the Value of Human Partner in Collaborative Learning
N = 20 CodingThis study in the framework
Not reported and not measured. The AI is a GPT-4-mini assistant giving hints, step-by-step to-do lists, code-run feedback and on-request code-block analysis, deliberately withholding complete solutions. It makes no prediction with a ground truth, so there is no correctness rate to report, and the paper reports none.
- RQ1: What are the benefits and drawbacks of human-human-AI triadic programming compared to human-AI pair programming?
- RQ2: How does AI positioning (shared vs. personal) affect learning experiences in human-human-AI triadic programming?
- RQ3: What types of AI support are useful for human-human-AI triadic programming?
"triadic settings activate socially shared regulation of learning by making AI use visible and accountable to a human peer"
abstract
"If I see that someone is watching my AI prompts, I would feel that I need to be more mature and try to understand the code first, rather than just copypasting."
results, 5.2.1, participant P6
"the presence of a human partner encouraged them to discuss solutions and use AI suggestions more responsibly"
discussion, 6.2
"in collaborative programming, effective augmentation means designing AI to reinforce the collaborative mechanisms that human partners already provide"
discussion, 6.1
Experimental design
Within-subjects study with three conditions, order counterbalanced.
Task. Three programming problems adapted from LeetCode, each with sub-tasks at varying difficulty, chosen to admit several solution approaches and to require algorithmic reasoning. 17 minutes were allowed per task.
Conditions. HHAI-shared: the two humans work together with a single shared AI acting as a third collaborator, addressing the pair collectively so that every AI exchange is visible to both. HHAI-personal: the two humans work together but each has an individual AI giving personalised, text-only guidance, so AI exchanges are private. HAI, the baseline: each human works alone in a separate space with an AI assistant.
AI agent. A GPT-4-mini-backed assistant that intervened proactively after five seconds of silence, at most once every 20 seconds, and could also be summoned by voice or text ("Hey Bob", "@AI"). It scaffolded with step-by-step to-do lists rather than complete code, analysed code-run feedback, and could analyse a selected code block on right-click. Its design goal was to give hints without revealing solutions prematurely. Participants were told nothing about the agent's reliability or limitations.
Measures. The Collaborative Learning Scale and the Social Presence Questionnaire; perceived responsibility for understanding AI suggestions; the proportion of AI-generated code adopted, counted manually as lines fully copy-pasted; AI suggestion uptake, annotated manually as suggestions incorporated, discussed or followed up; perceived AI disruptiveness; number of sub-tasks completed; and a coded analysis of conversational utterances.
Full findings
Putting a human partner beside the AI cut reliance on AI-generated code by more than an order of magnitude, and it did so through peer visibility rather than through any change to the AI.
The proportion of AI-generated code in participants' work fell from 23.1% in the dyadic HAI baseline (95% CI [4.4, 66.5]) to 1.4% in HHAI-shared (CI [0.2, 9.5]) and 1.2% in HHAI-personal (CI [0.2, 8.9]); Wald chi-square = 48.4, p < .001. AI suggestion uptake fell in the same direction and separated the two triadic conditions: 83.1% in HAI (CI [77.6, 87.4]), 67.2% in HHAI-shared (CI [59.0, 74.5]) and 45.0% in HHAI-personal (CI [37.6, 52.6]), Wald chi-square = 107.2, p < .001, with all pairwise contrasts significant.
The mechanism the authors propose is socially shared regulation of learning: making AI use visible to a peer makes it accountable. Perceived responsibility to understand a suggestion before applying it was far higher in HHAI-shared (M = 6.05, CI [5.61, 6.49]) than in either HAI (M = 2.80) or HHAI-personal (M = 2.60), Wald chi-square = 24.4, p < .001, and the shared and personal conditions did not differ from each other on that measure. That pattern is the evidence for the mechanism: what raises felt responsibility is not having a partner in the room but having the AI exchange itself visible to that partner. A participant states it directly, saying that if someone were watching their AI prompts they would feel the need to understand the code first rather than copy-pasting.
Collaborative learning and social presence both rose in the triadic conditions. Collaborative learning gave F(2,37) = 7.31, p < .01, with HHAI-shared against HAI at beta = 0.93, t(37) = 3.81, p < .001 and HHAI-personal at beta = 0.55, t(37) = 2.28, p = .028. Social presence gave F(2,37) = 9.45, p < .001, with the two triadic conditions not differing from each other. Talk changed in kind as well as volume: average utterances were 144.3 in HHAI-shared and 133.6 in HHAI-personal against 35.9 in HAI, and where HAI conversation was dominated by information-seeking questions (35.10%), the triadic conditions carried more acknowledgment (14.44 to 16.42% against 6.69%) and justification of proposals (2.62 to 5.34% against 0%), chi-square = 176.45, p < .001.
The cost is legible too. HHAI-shared was rated far less disruptive (M = 2.71) than HAI (M = 6.01) or HHAI-personal (M = 6.18), Wald chi-square = 23.5, p < .001, which the authors attribute to the personal AI fragmenting the pair into parallel private interactions and breaking synchrony. Task performance did not differ: sub-tasks completed averaged 1.7 in HAI, 1.5 in HHAI-shared and 1.4 in HHAI-personal, with no significant contrasts. The reliance reduction therefore came free in throughput terms over a 17-minute window, but no measure of solution correctness or code quality was collected, so the claim that reliance fell without cost rests on quantity completed rather than on quality produced.