Judgement over tokens

Anyone can prompt an app into existence now. I do it every day. That was never the hard part.

The hard part is knowing when the machine is wrong. That is the whole job, and it is the thing this site is named for.

The machine sounds the same when it is wrong

An AI model does not hesitate. It does not say "I think" in a worried voice. It tells you the feature works in the same tone it uses to tell you the sky is blue.

Here are three things an AI told me with a straight face.

• "All tests pass." Every automatic check was green. I used the real thing anyway, the way a customer would, and found about fourteen more bugs.
• "99.96% of records are populated." It had counted the placeholder word "auto" as a real answer. The number was wrong, and it was already sitting in the write-up the team was about to read.
• "There's a 72-hour grace period." There isn't one. It invented a billing rule and wrote it down like a fact.

None of those came with a warning label. Each one looked exactly like a correct answer. If I had not gone and checked, each one would have shipped.

This is not me picking on one bad tool. The people who build these models say the same thing. Anthropic's own write-up of one of its models reported that it sometimes edited the tests so the code would appear to pass. In the 2025 Stack Overflow survey, 66% of developers said AI output is "almost right, but not quite."

Almost right is the expensive kind of wrong. Completely wrong gets caught. Almost right gets shipped.

What throwing tokens at it looks like

A token is the unit an AI bills you in. Roughly, a piece of a word. Every prompt costs some, and every answer costs more.

When something breaks and you can't tell why, the cheap move is to spend more of them. Ask again. Ask a bigger model. Paste the error back in and say "fix it." Paste the next error in. Ask it whether its own fix is correct, and feel better when it says yes.

That is not building. That is pulling the lever again and hoping. If you can't tell when your AI is full of it, more tokens only get you more of what you already can't check.

What judgement looks like

Judgement is not a feeling. It is a set of habits, and you can watch somebody do them. These are mine.

1. Passing checks are necessary. They are not sufficient. Green lights mean the software did what the test asked. They don't mean the test asked the right thing. I use the product like a customer before I call it done.
2. Verify as you prompt. Prune as you prompt. I check each step when it lands, not at the end. And I throw out what is wrong right away, because an AI will happily build ten more floors on a cracked one.
3. The scratchpad is not the document. The AI's working notes are full of guesses. Nothing moves from the notes into something a person will rely on until I have checked it.
4. Five cheap narrow agents beat one long tired thread. A long AI conversation gets sloppy, the same way a person does at hour eleven. Small jobs with clear edges get better results and cost less.
5. Nothing ships until a person who can defend it says so. That person is me. If you ask me why something was built the way it was, "the AI did it" is not an answer I am allowed to give.

I vibe-code too

I am not here to tell you AI is bad. I would be out of work. Agents write most of the code I ship, and they are very good at it.

I also vibe-code, which is the nickname for letting the AI drive and not looking too hard at the result. There is a time and a place for it. A weekend prototype. A throwaway script. A game for my kid.

Production code that ships to your users or your customers and makes your first impression is not it.

What you are actually paying for

The typing got cheap. That is why my prices are low. I am not billing you for a week of keystrokes that a machine now does in an afternoon.

What stays expensive is the judgement. Deciding what gets built. Reading what came back. Catching the 72-hour grace period before it lands in a customer email. Putting a name on what ships.

You get the speed of the tools and the accountability of somebody who has done this before. I built the product at two companies that were later bought, and neither of those happened because somebody hoped.

If you have a product in mind, or one that is half built and making you nervous, book the free half hour. I'll tell you straight whether you need me.