Reading about GenAI tutoring

In Kestin et al. (2025), discussed in yesterday’s iEarth journal club (pity I missed the discussion!), an “AI-powered tutor” is compared against an active learning class, and they find that “students learn significantly more in less time when using the AI tutor, compared with the in-class active learning. They also feel more engaged and more motivated“. This is then interpreted as “empirical evidence for the efficacy of a widely accessible AI-powered pedagogy in significantly enhancing learning outcomes, presenting a compelling case for its broad adoption in learning environments“, so let’s see!

What is interesting in this study is that rather than using GenAI in whatever way they want (which usually does not work out so well, see for example Liu et al. (2026) or Poulidis et al. (2025)), students in this study work with an AI tutor that is specifically designed to promote learning. Kestin et al. (2025) list best practices from the literature that foster student learning, e.g. self-paced and active learning, scaffolding and managing cognitive load, making sure that information and feedback are accurate, and a growth mindset, and “aimed to design an AI system that conforms to these practices to the fullest extent current technology allows“. They share the prompt in the supplementary material, and it’s basically the instruction to be helpful and friendly (personalised feedback?), use emojis, and not to give students answers to the whole problem at once (manage cognitive load?), but rather help them with the next step if they ask (self-pacing) and encourage them to try themselves (growth mindset), and then there is some stuff about the tasks to make sure that the outputs are accurate. Kestin et al. (2025) write that they “believe that the increased learning from structured AI tutoring is largely due to its ability to offer personalized feedback on demand—just as one-on-one tutoring from a (human) expert is superior to classroom instruction“. They recommend that “an AI tutor should not replace in-person teaching—rather, it should be used to bring all students up to a level where they can achieve the maximum benefit from their time in class“, and something about that rubs me the wrong way.

About the self-pacing, they find that students who reported that the in-person classes moved too slowly spent less time learning with the AI tutor, whereas those that said that the lessons moved too fast spent more time, so that seems good that they got to do that.

Another article that looks at different ways to use an AI tutor is Brcic & Frljic (2026), “The Effortless Trap: Productive Struggle, AI, and the Illusion of Learning“. As also Kestin et al. (2025), they start out by praising what a human tutor can do, and set the goal to create similar learning conditions using AI. They provide examples of exceptionally successful tutors that “show what unusually patient, individualized, demanding teaching can do when it reaches the right learner at the right time. That is why AI matters for education: it may make such conditions less rare.”

They posit that — with or without GenAI — learners need to go through these steps in this order: Prime (motivating why students should care), Probe (first hard attempt), Point (teacher guides with questions), Attach (teacher provides one clean example), Strengthen (by showing more examples, returning to the topic later, teaching the idea to someone else, …), and Test (the final, unaided check).

Adding GenAI into this framework, they state that “AI has two legitimate jobs. First, it cuts busywork that is not the point, so the student’s effort goes to the real thinking. Second, it adds feedback and practice density that no single teacher could supply at scale.” But it must never replace students’ struggles: “if letting AI in makes the task feel effortless, it is in the wrong place” (my emphasis, or as others have said “no forklift in my gym!”). From this, it becomes clear that how GenAI should be used depends on the skill being learnt and practised, and cannot be decided on course level. Using their model, they state what that means in detail (plus they provide a lot of evidence from their literature, check out their Table 1 and 2!):

• Prime: the teacher motivates why students should care, but also “signals belonging”. GenAI can support by drafting hooks or recall warm-ups
• Probe: the teacher gives a hard task and withholds the answer for productive struggle. GenAI: “Stay out.” (emphasis in original)
• Point: teacher guides with questions. A guarded GenAI tutor can also provide hints, point to errors, etc (so this would be the personalised feedback)
• Attach: the teacher provides one clean example at the right time. GenAI could also provide more clean examples on demand
• Strengthen: the teacher gives a different problem and students explain it to each other. GenAI could be the tutee here rather than the tutor, but it is important that the student keeps doing the work
• Test: the teacher runs a “fair, secured, unaided check”, GenAI stays out of delivery (but might draft items that the teacher verts before deployment)

They then apply this model to the example of survivorship bias (the airplanes that came back with the holes in the non-critical places…). This is a really interesting and fun article to read, even though they are careful to point out that it is a “practical synthesis”. Sometimes those are exactly what we need!

And a third GenAI-tutor paper: Bastiani et al. (2025) on “Generative AI without guardrails can harm learning: Evidence from high school mathematics“. Bastiani et al. (2025) look at a highschool in Turkey and compare the impact of studying maths without GenAI with access to standard GPT-4 with specialised tutors, all in addition to their textbook and notes, on an exam without any resources. They find that relative to non-GenAI students or the GenAI tutor students, the ones with standard GPT-4 do 17% worse. This is both because they might receive inaccurate responses to their questions, and because they might be using it as a “crutch” to ask for solutions rather than for learning (which, in the GenAI tutor condition, isn’t a concern, because it is prompted to avoid both specifically).

And last one for today: Wang et al. (2024) on “Tutor CoPilot: A human-AI approach for scaling real-time expertise“. This is a massive study on 900 tutors and 1800 students, and here it’s the tutors that work with GenAI to improve their work with the students. The Tutor CoPilot reads the chats and then outputs several suggestions using strategies like “ask a question”, “provide a hint”, “simplify the question”, “affirm a correct answer”, “encourage the student”, that the human tutor can decide to use or not, or edit to whatever they like. They find that “tutors with access to Tutor CoPilot are more likely to use strategies that foster student understanding (e.g., asking guiding questions) and less likely to give away the answer to the student, aligning with high-quality teaching practices“, and especially tutors that were initially less effective benefit from working with Tutor CoPilot. That reflects in the results: students whose tutors had access to Tutor CoPilot were 4% more likely to pass the exit ticket. Human tutors themselves describe working with Tutor CoPilot as “helpful,
particularly for its ability to provide well-phrased explanations and break down complex concepts on the spot“, even though sometimes explanations were still too difficult for students.

This is a really interesting study and a good use of GenAI for learning — at least the students’ learning. It would be very interesting to see what happens with the tutors: Do they start depending on Tutor CoPilot instead of thinking themselves? What does it do to how meaningful their work is, how authentic their interactions with students, …? Also, did students know that their tutors were using GenAI in this way, and what would they think about that? So many open questions!

Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122.

Brcic, M., & Frljic, S. (2026). The Effortless Trap: Productive Struggle, AI, and the Illusion of Learning. arXiv preprint arXiv:2606.26181.

Kestin, G., Miller, K., Klales, A. et al. AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Sci Rep 15, 17458 (2025). https://doi.org/10.1038/s41598-025-97652-6

Wang, R. E., Ribeiro, A. T., Robinson, C. D., Loeb, S., & Demszky, D. (2024). Tutor copilot: A human-ai approach for scaling real-time expertise. arXiv preprint arXiv:2410.03017.

Sunset walk yesterday afternoon…