Does AI Help or Hurt Students? Professor Found Out with Two Exams

Does AI help or hurt students? One professor ran an experiment. Let's find out more in today's presentation. We all know Gen AI is totally changing how we learn and work, but most importantly for us today, it's upending how we actually test what someone knows. We've heard all the theoretical debates, right? But today, we've got hard data – a concrete example. Let's walk through this wild timeline and see what it actually means for the future of education. So let's dive into this crazy mystery that kicked off in a Brown University classroom this past spring. The course, Welfare Economics and Social Choice Theory, taught by Professor Roberto Serrano. Now he's been teaching this exact class for almost 20 years. He knows the material inside and out, he knows where students usually stumble, and he knows exactly what the typical grading curve looks like. But this spring, something totally unprecedented happened. He handed out an exam, and the scores that came back, they were so high they basically broke his grading scale. 96%. That was the staggering average score on this notoriously difficult take-home midterm. And it wasn't just a high average either. Out of the whole class, 40 students got absolutely perfect scores. To put that into perspective, getting just one or two perfect scores on a brutal college -level economics midterm is huge. But 40? I mean, that is literally a statistical earthquake. No way that just happens on its own. Looking at the historical data, the average for this midterm usually sits right around 65 to 80%. But here's the absolute kicker. Professor Serrano actually designed this year's test to be harder than the ones he'd given in the past. Since the students had unlimited time at home, he really wanted to challenge them. So you have an objectively harder test, but a massive, unprecedented spike in grades. That's a huge red flag that demanded a much closer look. Now, to be totally fair and understand the context here, we have to point out that this take-home format wasn't his usual procedure. It was actually a really compassionate move to help students who were dealing with a recent tragedy on campus. Back in December, there was a devastating mass shooting at Brown. Understandably, a lot of students were dealing with severe anxiety about sitting in a physical classroom for a high-stress test. So Professor Serrano made what seemed like the most humane choice. He let them take the exam from the safety of their own homes. Part 1 – Investigating the AI Suspect So faced with this super-difficult test, and a class size that had skyrocketed from the usual 30 students up to 86, Professor Serrano got suspicious. He started plugging his own exam questions into ChatGPT. And well, he spotted the AI's fingerprints almost instantly. For one specific question, you'd normally expect a standard human approach – a straightforward, direct argument. But instead, dozens of students submitted this highly convoluted, contradiction argument that perfectly mirrored ChatGPT's output. Suddenly, the professor is staring at dozens of exams using this identical, highly robotic logic. That brings us to Part 2 – The In-Person Final Acting on those glaring similarities, Professor Serrano essentially set a trap. He announced a pivot – the final exam was going to be proctored in-person, back in the classroom. And if the final grades didn't roughly match that amazing midterm average, the midterm would be totally voided. The reaction? It was practically instant. 18 students straight-up dropped the course. Just gone. And then another 9 students who stayed enrolled simply didn't show up for the final. So right out of the gate, nearly a third of the class noped out the exact second the test was moved to a supervised room where they couldn't access genderative AI. For the students who did show up to that supervised final, the results were, well, incredibly revealing. The class average absolutely plummeted from that 96 % high all the way down to a historic low of 48.6%. Think about that drop. Historically, the final exam average in this class had literally never dropped below 65%. Three students even scored flat zeros. True to his word, Professor Serrano voided the midterm, made the final worth 80% of the grade, and even lowered the passing threshold to 40%. But even with that generous curve, 19 students still failed the course outright. Now while I drop that drastic strongly points to widespread AI cheating, we have to look at this objectively. Universities can't actually rely on score drops alone as definitive proof. It's a really important nuance. Yes, unauthorized AI assistance on the midterm is a huge factor here. But you also have to consider the environment. Switching from an untimed, take-home test to a timed, high-stress classroom setting inherently hurts performance. Plus, you have the heightened test anxiety from the campus context we talked about earlier, and the loss of peer collaboration. That's exactly why schools can't just use score drops as a smoking gun. They have to rely on formal academic integrity processes, not just assume the numbers tell the whole story. Which brings us to part three, the institutional dilemma. So when Professor Serrano took this data to Brown's Academic Code Committee, he actually refused to use standard AI detection tools, because as we know, they're notoriously prone to false positives. But that creates a massive headache. How do universities manage mass allegations of cheating without those tools? The reality is, the system just wasn't built for this kind of scale. As academic integrity expert Trisha Bertram-Gallant from UC San Diego points out, faculty just aren't incentivized or compensated to deal with massive, complex cheating cases. I mean, think about a grad student or a lecturer whose pay stops on the last day of class. When are they supposed to process 60 individual, highly nuanced academic integrity violations? Making mass reporting a requirement just turns into a logistical nightmare, and a lot of times these cases just stall out completely. And this brilliantly illustrates the macro issue here. Bertram-Gallant brought up this great quote from organizational theorist Edgar Schein, basically saying that leaders show what they value by what they spend their money on, not just what they say. It hits the nail on the head. If universities truly value academic integrity in the generative AI era, they can't just fire off strongly worded emails. They have to put actual funding and dedicated, paid staff behind it. The fact that a lot of wealthy universities don't even employ dedicated academic integrity professionals, that's a pretty glaring gap in the system. And finally, part four, the future of assessment. Zooming out a bit, Brown University's own AI committee found that 75% of faculty are worried about AI cheating, which is forcing institutions everywhere to totally rethink how they assess students. They're updating academic codes and posing some rapid -fire, deeply philosophical questions. For example, is a piece of writing truly in your own words if you wrote a rough draft but an AI provided all the grammatical improvements and polish? Where exactly is that line between using a modern digital tool and committing academic dishonesty? Or, let's take it even a step further. What if you bounce ideas off an AI to generate a thesis exactly like you would with a human study partner at a coffee shop? Who actually owns the resulting thoughts? How do we apply the old rules of original research when your collaborative partner is a machine learning model? These are the crazy gray areas that professors and administrators are desperately trying to figure out right now. Because 100% accurate AI detection is basically impossible, the whole educational landscape is having to shift its strategy. According to the committee, schools actually need to avoid highly restrictive draconian rules around AI, and even de-emphasize purely punitive measures. Instead, we have to embrace changing tech norms. Moving forward to measure what a student really knows, we're probably going to see a massive shift back to supervised in-person exams, oral presentations, and specific project-based evaluations where you have to actively prove your knowledge. We're going to wrap up this explainer with a pretty stark warning from Professor Serrano about the long-term societal cost of all this. He warns that outsourcing our thinking leads to a declining society, saying point-blank, we cannot choose to become idiots. It is a really powerful reminder. AI is an absolutely incredible tool, for sure, but the core purpose of education is still human cognitive development. Which leaves us all with a pretty heavy question. Moving forward, how do we integrate this amazing tech into our schools, without losing our own fundamental ability to think, reason, and solve problems for ourselves? Thank you so much for exploring this with me today.