Margin notes on Karpathy.

Andrej Karpathy did a talk this month called From Vibe Coding to Agentic Engineering, and I watched it with my own blog open in another tab the whole time, half because I was bored and half because I kept hearing sentences I'd already pencilled here in different clothes. The talk's worth twenty minutes of your time before reading the rest of this; the embed is below.

What follows is a set of his moments that struck me, sitting alongside the posts here that already touched the same nerve, plus one or two places where I'd push past where the talk stops.

That December moment is exactly the inflection a generation of working programmers walked across without quite naming it. The thing Karpathy is describing, the moment you stop reaching to correct the model's chunks and start trusting them, is a quiet handover of the job from generation to review. I wrote about this same shift from a different angle in the post about coders teaching the model to ship, where the analogy was a tabla ustad in a Mirpur classroom who claps the pulse and listens but never plays the drum during the lesson. The apprentice does the playing. The ustad sits with a manuscript on his lap and a piece of pencil in his hand and corrects from the wrist. Karpathy walked into that room in December and noticed the apprentice no longer needed his hands on the rosewood at all.

What he calls Software 3.0, the shift from writing code to writing prompts that get interpreted by the LLM, lands in his MenuGen story with more force than the tidy diagram that lays it out in three stages. He'd built an OCR app that takes a photo of a restaurant menu and renders pictures of each dish. Then Gemini and Nano Banana came out, and the same job took one prompt and one image, no app at all.

The moment the artifact you built last year stops needing to exist is the moment a lot of product work has to be examined all over again. It's the same shape I traced in the post about the death of the PRD, set in a Sutrapur notary office where the document used to be the contract and is now mostly a receipt for one. The PRD didn't get worse over the last two years; the thing it was scaffolding around quietly collapsed into the model and made the scaffolding redundant. MenuGen is the same shape. So is, I think, half of what the average product team still ships every quarter under the heading of "platform tooling."

The verifiability section is where Karpathy is most useful and most quietly irritating, because he describes the structure of the problem cleanly and then politely refuses to name the consequence. He says traditional computers automate what you can specify in code, and LLMs automate what you can verify, because the labs train them with reinforcement learning over verifiable rewards. So the models peak in math and code, where the reward is unambiguous, and stay jagged everywhere else. The example he keeps using is the letter count in strawberry, replaced now by the car wash one.

Right, but that's also the whole story of benchmark drift. When I wrote about the half-life of benchmarks I was sitting in Sajjad bhai's recording studio in Mirpur, watching him grade song demos against a scale of his own making, built in 2003, which by 2026 predicted nothing about which demos would chart. Jaggedness is the same phenomenon dressed in different clothes. A model that scores 96 on MMLU and tells you to walk to a car wash hasn't gotten worse at MMLU; the test stopped being the thing. Verifiability tells you which test the labs decided was worth gaming hard during the last training run, and the jaggedness everywhere else is the residue of every domain they didn't bother with. The question worth asking each quarter is whose domain the model is jagged in this time round.

The line that made me put the pencil down for a second came near the end, in his answer about education and what's still worth learning when intelligence gets cheap.

I sat with that for about twenty minutes. It is, more or less verbatim, the argument I'd been trying to make in the post about confusing thinking with programming. The analogy there was a generation of Bangladeshi schoolkids who could pass any English grammar exam the country put in front of them and still couldn't order a meal at the airport at twenty without sweating through a polyester shirt. The grammar of any subject (code, English, music, the violin, double entry bookkeeping) is the thing you can outsource to a tutor or a syllabus or an LLM, and what comes back is the paper score the syllabus was designed to give you. The understanding the score was supposed to measure stays in the room with you, or it doesn't, and no amount of outsourcing will move it. Karpathy's sentence puts the same point in a tighter envelope, almost the same week as a draft I'd been pushing around with the same fountain pen.

There are two places where I'd push, gently, past where Karpathy stops. The first is on taste. He says people are in charge of the taste, the judgment, the spec, and the question he politely declines to answer is which people. The default reading, in San Francisco, is senior engineers with strong opinions about brittle abstractions. I think that's half the answer at most. The other half, the half I wrote about in the thinking-vs-programming post, is that the deciding skill has lived for a long time outside the engineering room. The senior CSM who has watched seven customer teams break the same product seven different ways already carries the whole catalogue of failure modes in her head. So does the implementation architect with forty onboardings behind him, who can usually call which two integrations the new prospect will fight in week three before the kickoff call ends, and so does the support lead whose two hundred tickets a quarter teach her exactly which line in the error log means the auth handshake stalled. None of those people ship a line of TypeScript on a Tuesday morning, but give them a model that ships the TypeScript for them and they will make sharper calls than the engineer who has never opened a Zendesk queue. The sentence Karpathy declines to finish is which people, by name, the org chart should now be listing as eligible to direct the taste, because at most enterprise software companies in 2026 the answer is still written the way it was written in 2018.

The second push is on agents being statistical. Karpathy describes them as ghosts rather than animals.

The conclusion he draws is correct, that yelling at them doesn't help. Where the talk stops is at the testing question, which is what I'd been chewing on for the post about agentic contract tests. If the agent is statistical, the test for it has to be statistical too, which is roughly the move a kabin-nama makes when a Tejgaon kazi draws one up. The kabin-nama doesn't promise that two specific people will behave a particular way under it; it promises that whoever marries under its terms will be bound by these shapes of obligation. Shape contracts written the way Pact writes them, where you assert the structure of the response rather than its exact text, are the testing pattern that fits the statistical reality of agents. This isn't a correction to Karpathy so much as the sentence I'd add in pencil at the bottom of his page.

What I keep coming back to, after watching the talk a second time, is how unsurprising the overlap is. The shift Karpathy is naming has been visible for a while from inside an enterprise account that ships agentic features to regulated customers, and watching him put it in a tighter envelope clarifies what's worth writing next. Two posts are already sitting half written in the queue, one on who exactly gets to direct the typist now that the engineering org charts have to be reread, and another that picks up the thread on testing contracts for stochastic agents in more technical detail.