AI tutoring

AI tutoring is the use of software, from rule-based intelligent tutoring systems to large language model (LLM) chatbots, to teach a person: choosing what to present next, checking whether the learner understands, and responding to their work. The central dispute is whether general-purpose chatbots help or harm learning when the learner is later tested without the AI. Evidence from one large school experiment says unrestricted access can harm (Bastani et al., 2025), a hint-giving tutor removes the harm but has not been shown to add benefit (Bastani et al., 2025), and purpose-built tutors have produced gains in some trials (Kestin et al., 2025). Most of the evidence is recent, much of it preprints, and much of it measured while the AI was still available.

History

Step-based intelligent tutoring systems (ITS), which respond to each step of a problem rather than only to the final answer, were reviewed against human tutoring by VanLehn, who found them nearly as effective as human tutors and so contradicted the belief that finer-grained interaction always yields larger gains (VanLehn, 2011). A line of Cognitive Tutor experiments added worked examples to tutored problem solving and faded them as learners improved (Salden et al., 2010). LLM tutors arrived later: one early study compared ChatGPT-written algebra hints with human-written hints in 2023 (Pardos & Bhandari, 2023).

A fuller history (the first ITS, the move from rule-based to LLM tutors, vendor tutoring modes) is missing from the research.

Core concepts

Sequencing material

Sequencing guidance for AI tutors comes mostly from earlier learning science rather than from tests of LLM tutors. The expertise reversal effect says that techniques that help learners with little prior knowledge can be less effective for those with more, so instruction should be adjusted as knowledge grows and worked-out steps faded as understanding is shown (Wikipedia contributors, 2026d) (Wikipedia contributors, 2026d). In two experiments with a Cognitive Tutor, adaptive fading of worked examples gave better learning than fixed fading, which in turn beat problem solving (Salden et al., 2010). Earlier work found that gradually fading worked-out steps into problem solving beat example-problem pairs, at least for near transfer, with fewer learner errors (Renkl et al., 2000). Studying worked examples, or an example followed by a practice problem, was more effective and efficient than problem solving alone (van Gog et al., 2010). Self-study materials covers these effects in more detail.

For multimedia lessons, learner-paced segments helped in 10 of 10 tests (median effect size 0.79), and teaching the names and characteristics of main concepts before the lesson helped in 13 of 16 tests (median 0.75); these results are from multimedia lessons and were not tested in AI tutors (Mayer & Pilegard, 2014).

Two designs address diagnosing what the learner knows. One LLM tutor design traces prerequisites recursively with binary knowledge checks to find the learner’s knowledge boundary, but it has only been demonstrated, not tested with learners (Tang et al., 2025). A deep knowledge tracing model outperformed zero-shot and fine-tuned LLMs at predicting next-step correctness, and the authors conclude that LLMs alone are unlikely to match established ITS at learner modelling, estimating what a learner knows (Hooshyar et al., 2025). In a five-month high school Python course with an AI tutor, an adaptive problem-sequencing algorithm raised unassisted final exam performance by 0.15 standard deviations over a fixed sequence (preprint) (Chung et al., 2026).

Checking understanding

Two preprints test checks of understanding with LLM feedback. In a calculus experiment (N = 92), requiring open-ended self-explanation with LLM feedback raised explanation quality on one transfer problem type, by about 11.9 points (p = .030), but post-test performance did not differ between conditions (Chen et al., 2026c). A quasi-experimental study of Socratic Mind, a generative-AI formative assessment tool, reports that students who used it gained more on quiz scores than those who did not, especially students with lower baseline achievement (Lee et al., 2025c). In a small preprint (about 60 college students), a week with LLM-generated multiple-choice retrieval practice produced higher quiz accuracy than a week without it, though the question quality varies and the authors advise checking questions first (An et al., 2025). Practice testing and distributed practice were rated highest in utility in the Dunlosky et al. review of ten learning techniques (Association for Psychological Science, 2013b); the Transfer of learning article treats retrieval practice more fully.

Feedback

Hattie and Timperley’s model frames feedback as three parts: feed up (the goal), feedback (progress toward it) and feed forward (next improvements) (Ellis & Loughland, 2017). It distinguishes four levels of feedback, task, process, self-regulation and self, of which the first three are described as most beneficial (Smit, 2023). The empirical results for AI feedback are small and mixed.

In a randomized comparison with 459 upper secondary EFL students, LLM-generated feedback gave a small gain in essay revision versus no feedback, measured with the feedback in hand rather than as later learning (Meyer et al., 2024). In a study of on-demand LLM feedback in tutor-training lessons, only 2 of 7 lessons showed significant learning benefits after propensity adjustment, and benefit depended on learners choosing to use the feedback (Thomas et al., 2025). An audit of 2,389 GPT-4 feedback messages on statistics exams found about 7% contained errors, from minor slips to conceptually misleading explanations (Herklotz et al., 2025).

On the form of hints, an early algebra preprint (n = 77) found human-written hints produced larger gains than ChatGPT hints (Pardos & Bhandari, 2023). The published follow-up (N = 274) found only ChatGPT hints produced significant gains over a no-help control, and ChatGPT and human hints did not differ significantly; ChatGPT help failed quality checks on 32% of problems until a self-consistency filter was applied (Pardos & Bhandari, 2024).

What vendors say tutoring modes do

Developers describe their tutoring modes in similar terms. Anthropic describes its Learning mode as guiding reasoning with Socratic questions instead of giving answers (Anthropic, 2025b). A developer-extracted version of the ChatGPT study mode prompt tells the model to ask about the user’s goals or level first, to guide rather than give answers, and not to do the user’s work (Willison, 2025). Google describes Guided Learning in Gemini as teaching through probing questions and step-by-step explanations, and says prompting alone was not enough to build it (Heymans, 2025). Google’s LearnLM authors translated learning-science principles into seven educational benchmarks and name the lack of good evaluation practices, and the difficulty of defining excellent pedagogy, as a key obstacle (Jurenka et al., 2024). The announcements for Anthropic’s and OpenAI’s modes report no evaluation of their effect on learning (Anthropic, 2025b) (Willison, 2025).

Questions, controversies, and new developments

Does unrestricted chatbot access harm learning?

The main evidence is a field experiment of nearly 1,000 students in grades 9 to 11 at one large Turkish high school, with a closed-book, closed-laptop exam (Bastani et al., 2025). Students with unrestricted GPT-4 (GPT Base) scored 17% lower than the control group on the later exam, a statistically significant drop (Bastani et al., 2025). A GPT Tutor variant whose prompt told GPT-4 to give hints without the answer, with correct solutions and common student mistakes supplied, removed that harm but showed no positive exam effect compared with control (Bastani et al., 2025). It nonetheless raised practice scores by 127%, and its students did not score better on the exam yet perceived that they had performed significantly better (Bastani et al., 2025). The authors state the study covers two tutors, one topic (mathematics) and short-term outcomes only (Bastani et al., 2025).

Other studies do not settle the matter. A randomized crossover trial in a Harvard introductory physics course (194 eligible of 233 enrolled) found students learned significantly more from a purpose-built AI tutor than from in-class active learning; the tutor walked students through each problem part by part and used pre-written solutions rather than relying on GPT-4 to produce them, but the comparison was not against an answer-giving AI (Kestin et al., 2025). In a randomized field experiment with more than 6,000 Tennessee middle-school students, delayed-test gains (one week) were only marginally significant and appeared only when the AI tutor was combined with mastery progression; the design compares the AI tutor with non-AI practice software, so it neither confirms nor refutes the 17% harm (Oreopoulos et al., 2026). In a proctored randomized experiment with 211 Middlebury undergraduates (AI allowed versus forbidden), AI access raised test scores and a smaller gain persisted at a one-week unaided follow-up; the gains held for students who used AI as a tutor and faded for students who had it write for them, a non-randomized split (Contractor & Reyes, 2026). In a four-week statistics quasi-experiment, students trained to use an LLM for stepwise hints showed only a “promising pattern” of better no-help quiz performance, so the benefit is tentative (Amanlou et al., 2026). A UK classroom randomized trial (exploratory) found students tutored by a pedagogy-tuned LLM, with expert tutors supervising each message, did at least as well as students with human tutors, and were more likely to solve novel problems on later topics (66.2% vs 60.7%) (LearnLM Team Google & Eedi, 2025).

A preprint meta-analysis finds LLM effects on student learning positive but uneven, strongest when LLMs act as tutors in sustained interventions and weaker for autonomy and learner development; it argues that evaluations often reduce impact to narrow performance metrics, so the outcome measure chosen shapes the reported effect (Huang et al., 2025).

Vendor-reported outcomes

Outcome evidence for vendors’ tutoring modes comes from the vendors. Google reports a randomized trial at the University of Tokyo (300+ students, half a day on a statistics lesson) in which Guided Learning users gained 8.1 percentage points against 5.3 for the control; no paper is linked and whether the post-test was AI-free is not stated (Cornelis & Johnson, 2026). A vendor-authored preregistered randomized trial in Sierra Leone (1,763 junior secondary students, 48 classrooms, 8 weeks) reports a gain of 0.258 SD (intent to treat) and 0.380 SD for those reaching 12 hours; the summary page does not give the outcome test or say whether it was AI-free (LearnLM Team & Fab AI, 2026).

Should the learner attempt the problem first?

Evidence on making the learner try before the AI helps points in different directions. In a randomized experiment, students who attempted a chi-square problem before using ChatGPT outperformed students who used it immediately on a later test (Akgun & Toker, 2024). A preprint of three studies found that attempting a task first improved learning, mostly on complex tasks and less on simple computational ones (Akgun & Toker, 2025). A productive-failure LLM tutor was tested with 17 students, and its abstract reports that the tutor followed the strategy, not that students learned more (Puech et al., 2025). Self-explanation before generative-AI use reduced transfer relative to PDF-only and AI-only study, especially for low-familiarity learners and with no clear reduction in illusion of knowledge (Lepennetier et al., 2026). In a CS1 trial, a prompting-skill intervention with LLM feedback gained most on a delayed post-test, but final exam scores did not differ between groups (Xiao et al., 2026b).

Feedback that demands reflection

In a large biology course, different types of GPT-4o practice-exam feedback did not differ significantly in performance; required confidence ratings and explanations had the biggest impact (Ahmad et al., 2025). A 329-student randomized trial found that students given metacognitive-only AI feedback revised least often (12.1%), against 21.1% for directive and 27.5% for hybrid feedback, but resource quality and engagement time did not differ, and the higher revision under hybrid feedback did not show up as an outcome difference (Alsaiari et al., 2025) (Alsaiari et al., 2025).

Illusion of competence and over-reliance

Novice programmers who struggled with generative AI often overestimated their performance and finished with an illusion of competence (Prather et al., 2024). In a study of AI-assisted logical reasoning, participants overestimated their performance by about four points and higher AI literacy was linked to less accurate self-assessment (N = 246 in the arXiv version) (Fernandes et al., 2024); a replication with a monetary reward for accuracy did not remove overestimation (Dolan, 2025). In a randomized study of 117 students writing an essay, ChatGPT support improved essay scores but knowledge gain and transfer did not differ from other groups, and the authors warn of dependence and metacognitive laziness (Fan et al., 2024). Proposed mitigations are mostly untested: post-task reflection and cognitive forcing prompts are suggestions (Fernandes et al., 2024). Cognitive forcing designs did reduce overreliance in AI-assisted decision-making (not learning) without eliminating it, but participants rated them least favourably and the benefit went more to people high in Need for Cognition (Bucinca et al., 2021). Sycophancy, a model’s tendency to agree with the user, is a related risk covered in Sycophancy in language models.

Do learners want the tutoring that teaches them?

Learners do not always favour guidance that withholds answers. In a graduate robotics course, learners rated a Socratic-guidance tutor as less efficient than a prompt-refinement tutor, yet its students later achieved higher learning gains (Brender et al., 2026). In a survey of 90 students and 50 educators on programming tools, 68% of students favoured direct help, educators leaned toward indirect scaffolding, and Socratic questions were among the least favoured scaffolds; the authors propose, without testing, progressive disclosure and capped assistance levels (Ma et al., 2026a). The gap between feeling and learning has a classroom precedent: in a randomized physics course study, students in active classrooms learned more but rated their own learning lower, and a roughly 20-minute talk explaining that feelings of learning can mislead was followed by 75% agreeing it helped (self-report) (Deslauriers et al., 2019).

Does personalisation help?

In a between-subjects online study (n = 272), generative-AI personalisation of vocabulary examples did not change learning performance but raised motivation (Leong et al., 2024). A preprint reports that LLM-personalised course materials improved outcomes over standard materials, without sample, randomisation or effect size reported (Lim et al., 2025), and an algebra ITS without LLMs found that personalising problems to interests affected classroom exam performance through situational interest (Bernacki & Walkington, 2018).

Adult self-directed learners

Research on adults learning outside a course is adjacent to AI tutoring rather than part of it. Empirical research on whether andragogy, the adult-learning framework that assumes adults are self-directed, improves learning remains inconclusive (Rachal, 2002). A 2023 meta-analysis reports that achievement rises with self-directed learning readiness (correlational) (Yurdal & Toraman, 2023), and a 2026 conference paper states that adult self-directed learners often struggle with persistence, proposing an AI coaching system but reporting only expert review (Yamamura et al., 2026).

Broader connections

How the tutor is presented, as a chat or a structured lesson, is the subject of Tutoring interfaces. The effects of worked examples, hints and self-explanation when no tutor is present are in Self-study materials, and whether learning carries into later use is the question of Transfer of learning. Whether explanations produced by an AI give the learner an accurate mental model is the subject of Explanation and understanding.

Further reading

• Bastani et al. (2025): the principal experiment on answer-giving versus hint-giving GPT-4, with an unassisted exam.
• Kestin et al. (2025): a randomized crossover of a purpose-built physics tutor against active learning.
• VanLehn (2011): the review comparing human tutors, intelligent tutoring systems and other tutoring systems.
• Contractor and Reyes (2026): an adult-age randomized experiment with a one-week unaided follow-up.
• Jurenka et al. (2024): how one vendor turned learning-science principles into evaluation benchmarks.