AI Tutor Khanmigo Fails to Engage Students Seeking Answers

A two-year study conducted by researchers from the University of Toronto has revealed that students largely disengaged with Khanmigo, an AI tutor developed by Khan Academy, when it was programmed to withhold direct answers and instead guide them through Socratic questioning. The study, which followed students in 18 Tennessee middle schools from 2024 to 2026, aimed to assess the effectiveness of AI tutors in improving the math skills of low-achieving students. These students were at least one grade level behind their peers and had an additional remedial math period integrated into their school day. The research, circulated by the National Bureau of Economic Research in August, indicated that while nearly all students initially tried Khanmigo, many attempted to bypass its tutoring function by asking for direct answers. When Khanmigo consistently refused and instead offered hints or posed questions, a significant portion of students ceased using the AI assistant. Philip Oreopoulos and Nina Low, the researchers behind the study, noted in their draft paper that "we observe how students actually used the AI tutor, and the answer is: not much." They further elaborated that "seeking help with one’s own confusion remained a choice, and most students declined it most of the time." The intervention group received instruction through Khan Academy with Khanmigo available as a supplementary AI assistant. In contrast, the control group continued with their standard remedial math curriculum, which often incorporated other online educational tools such as Waggle, IXL, Zearn, and DeltaMath. Despite the students' lack of engagement with Khanmigo's pedagogical approach, the students who participated in the Khan Academy intervention did demonstrate improved math performance compared to those in the traditional remediation classes. This suggests that while the AI tutor's interactive method was not favored by students seeking quick solutions, the overall Khan Academy platform, potentially augmented by the AI's presence, still offered some educational benefits. The findings highlight a critical challenge in educational technology: the necessity for student motivation and willingness to engage with learning tools, even when they are designed to foster deeper understanding rather than provide immediate gratification. The study underscores that the mere availability of an AI tutor does not guarantee its effective utilization if students are not predisposed to the learning process it encourages. The initial release of ChatGPT in November 2022 had already highlighted the propensity for students to use AI chatbots for completing assignments, a practice that was found to be detrimental to learning and resulted in lower test scores. Khanmigo was conceived as a response to this issue, aiming to create a more constructive AI interaction by emulating a human tutor's guidance.