DENVER, CO – MAY 2 : From left, Sally Wilson, 15, Reese Myers, 16, and 11th grade students discuss about essay for English class final exam at Thomas Jefferson High School in Denver, Colorado on Thursday, May 2, 2024. (Photo by Hyoung Chang/The Denver Post)
Denver Post via Getty Images
Easy access to AI answer bots is killing learning.
That’s one of the conclusions in a new and troubling study about AI use and education. The paper indicates that AI use by students in assessments is common, reduces learning investment, and significantly erodes actual learning.
In this study, students who used AI on math assessments faced a massive 25% learning loss since AI emerged. Learning loss is the lack of ability to answer questions correctly when they see them again.
The paper is titled, “Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build,” from authors Sina Rismanchian, of University of California, Irvine, and four researchers from McGraw Hill, the prominent education publisher and digital learning company. Those four are: Hasan Uzun, Jeffrey Matayoshi, Eric Cosyn, and Eyad Kurd-Misto.
The team studied student behaviors on a math assessment called ALEKS, “a widely deployed adaptive mathematics learning and assessment platform serving more than four million students annually.” Their observation window covered more than 3.2 million student interactions over more than a decade – before, and since the arrival of easily accessible AI bots.
The paper focuses on a measure known as time on task, how long a student spends on a particular question. For this, the research team looked at two types of math questions – word problems that are pretty easy to drop into an AI answer bot, and questions that are more graphic or image based, making them less vulnerable to low effort AI answering.
What they found was troubling.
For years before AI, time on task was steady. It did not vary much at all between the two kinds of math questions. But as soon as easy AI showed up, student behaviors changed. Time on task for the problems susceptible to AI answering has fallen since 2023. ChatGPT launched in November of 2022. Since AI, time spent on the AI-susceptible math problems has dropped 27% among college students. In high school, the drop is 31%. In middle school, time on task has declined 9%.
That decline is important because time on task is a key learning indicator. It shows how much effort a student is putting into understanding, thinking about, and trying to answer the question, how much thinking, reasoning, and calculating they’re doing.
Effort matters. Just like hitting the gym, stay longer, work harder, get better results. Before AI, let’s say that students were putting an hour in at this particular math gym, now high schoolers are clocking out after 45 minutes. Middle school students are logging out after 40.
As a result of less time working, students aren’t learning.
The report shows that students are now 25% less likely to get these math questions – the ones AI can punch out easily, the ones that students are spending less time on – correct. In other words, by using AI instead of doing the work, today’s students now know 25% less math than they would have before AI arrived.
This loss of ability, the paper says, “is the downstream consequence of the earlier learning phase, during which students appear to have relied on AI assistance to complete AI-susceptible problems without developing the durable understanding those problems were designed to build.”
Students took AI shortcuts to get answers and did not learn, in other words.
Rismanchian, of University of California, Irvine, said, “Settle on this fact, that students are using AI to bypass learning. Prior to this, it has not been conclusive. Stanford found that students self-reporting of cheating has been stable, pre and post AI – the numbers here are tracking – it’s something real.”
There are serious issues with that study from Stanford on rates of academic cheating after the wide availability of AI. Either way, what this new data shows is, as Rismanchian put it, “something real.” And probably serious.
Of further interest to some is that this new research across a decade and millions of data points is that proctoring the assessments – actually securing tests and being able to see or monitor student behavior during the questions – works to stop the cheating. It’s no surprise that students are less likely to cheat, to take AI-powered shortcuts, when they know someone may be watching.
“Students are behaving very different in proctored and non-proctored settings, in non-proctored settings, divergence [in time on task] is there. Even in high stakes tests, we saw this,” Uzun, one of the McGraw Hill team said.
Cosyn, also with the McGraw Hill team, said, “The cost of taking shortcuts is zero thanks to those new tools, but the effect vanishes with proctoring.” He added though that proctoring was, “Not a solution. You can’t do surveillance all the time,” he said.
Personally, I am not sure why not. If there’s a problem – and this new report shows a very clear, very serious problem – and you know what will fix it, it’s hard to understand intentionally not fixing it. Letting students continue to cheat, shortcut their learning, and endure sustained learning loss does not feel like the best option.
Returning to the headline, easy access to AI-powered cheating tools during exams is undercutting learning. Maybe that should have been obvious. But in case it wasn’t, it ought to be now.
What, if anything, we’re prepared to do about it is an entirely different question. One we may not want to nudge off to AI to solve.