I admit I’m angry.
I’m a professor at a major university teaching students to become the next generation of professional leaders. Yet instead of engaging with them as critical thinkers, it feels like teaching has become a game of whack-a-mole, in which students manipulate assignments with AI and then expect to be rewarded with remarkably good grades. When caught, the story always seems to be the same: deny or provide some convoluted explanation for how the error happened “by mistake,” and expect to rewrite.
A slew of new studies are providing evidence that this behavior is likely to have serious consequences longer term. My students may actually be becoming worse at critical thinking and less able to handle complex problems—and that is a harbinger of an impending crisis for their future employers and society at large. If they are uncritically submitting AI slop to their professor at graduate school, what can we expect them to do in an intense workplace?
We are a long (long) way from AI being able to really think and to judge the appropriateness of its actions. I’ve heard it said by Mohammad Jarrahi at the University of North Carolina at Chapel Hill, who studies human-AI interactions, that AI is really just a horse—a means to get somewhere quicker and with less energy. But the horse doesn’t understand where it is going or why. It knows only what it is being delegated to do.
The trouble is that many AI users have mistaken delegation for responsibility. When AI hallucinates and produces a list of fictional scientific papers to help pad a student’s reference list for an assignment, AI is just doing what it was told. The person issuing the commands must remain responsible.
And therein lies the problem. People are losing the ability to think for themselves. That might sound hyperbolic, except the evidence is mounting that this is in fact happening. There have been a number of studies which show that reliance on AI makes us worse problem-solvers and overly dependent on bad advice.
The many studies now appearing all tend to show very similar patterns. For example, a study by Shaowei Ren from the Anhui University of Science and Technology showed that among undergrads who participated in an experiment where they were asked to use ChatGPT, almost two thirds accepted incorrect advice and then underestimated how much AI had influenced their choices. Only when they were encouraged to pause and reflect on the validity of the AI advice did they better calibrate the accuracy of what they had been told.
In another experiment with high school students, access to AI improved performance substantially on mathematics—but students performed far worse than peers when access to AI help was withdrawn. Interestingly, Hamsa Bastani from the University of Pennsylvania found that when AI was used in partnership with teacher-directed instructional prompts that provided students with opportunities to learn problem-solving skills (rather than just providing students with answers), students not only performed better during the experiment, but also didn’t do worse after AI was withdrawn.
Over and over again, the story is the same: People who use AI frequently show a pattern of “cognitive offloading,” meaning their ability to think critically is diminished.
AI has its purpose, just as a calculator, the internet, and a horse have theirs. But there are consequences to giving up personal agency, both in the short and likely in the longer term. For the moment, given the warning signs, it is best we don’t ask “horses” to write university papers.
