Paul Graham’s X Post Raises a Bigger Question Than AI Cheating
TL;DR
Q: What happened?
A: A Brown econ class scored much higher on a take-home midterm than an in-person final. Graham’s take: most of the class cheated.
Q: What’s the real signal?
A: It’s about to get a lot harder to tell who understands the material from who just produced the right answer, in school and in hiring.
Q: What’s the fix?
A: Stop grading the output. Start measuring the reasoning behind it: can they explain it, defend it, catch it when it’s wrong?
Paul Graham posted a chart that’s been making the rounds: a Brown University econ class took a take-home midterm (then, after the professor suspected AI use, an in-person final). The scores dropped off a cliff.
His conclusion: most of the class cheated.
That’s a fair assumption. But we think it’s focusing on the symptom, not the signal.
As long as there have been tests, there’s been cheating. AI didn’t invent that. What it did is make cheating easier, harder to catch, and nearly impossible to distinguish from legitimate help.
The real signal is that it’s about to get a lot harder to tell the difference between someone who understands the material and someone who just produced the right answer.
That gap matters more, not less, as AI gets better. The people who can guide AI to its fullest capability are the ones who know the terrain underneath it.
A computer science student who understands systems at a full level, not just how to write code, can tell when an AI’s suggestion will break under load. A physics student who understands the underlying principles can catch it when a model quietly hallucinates a wrong assumption. Someone who only knows how to prompt can’t catch either one. They can only trust that nothing went wrong.
So the answer for schools isn’t banning AI or forcing everyone back into a blue book. Both are reactions, not solutions. The real work is redesigning how learning gets measured: more oral exams, more in-class problem solving, more assignments that ask a student to show their reasoning, not just their output. Let students use AI, then ask them to defend what it gave them. That is the test that actually shows who understands the material.
We’re asking the same question in hiring. Resumes, cover letters, coding exercises, writing samples, AI can touch all of it now. That doesn’t make them worthless. It makes them unreliable as standalone signals.
The real fix is building systems that assume AI is in the room and measure something AI can’t fake: the reasoning behind the output. Can someone explain their decisions? Challenge their own assumptions? Catch their own mistakes? Adapt when the situation changes? Do they know enough on their own to notice when the AI is wrong?
That’s harder to test than a take-home assignment. It’s also the only thing worth testing now.
The organizations that figure out how to measure that, instead of clinging to signals AI has already hollowed out, are the ones that’ll have an edge.