A Green Test Suite Is Not a Ship Decision

A change ships. The test suite was green. Two weeks later, checkout breaks for users on mobile Safari because the new submit handler never disabled the button, and a double-tap submitted payment twice.

Nobody lied to anybody. The tests passed. That's exactly the problem: a green suite proves the tests you thought to write pass. It doesn't prove the change works.

AI tooling has made implementation genuinely faster. I can produce five reasonable implementations of anything before lunch. What hasn't gotten cheaper is deciding which one is right — and verifying that the one I ship actually works under real conditions.

So as creation accelerates, verification becomes the bottleneck. Not because verification got harder, but because the volume of changes worth verifying went up. If your evidence process can't scale with your creation speed, you end up shipping unverified code and calling it velocity.

Here's the part that gets skipped: not every change deserves the same scrutiny.

A copy change and a checkout change are not the same kind of risk. One has a blast radius of a paragraph. The other touches money, state, and trust. Requiring the same heavy verification for both makes the process slow enough that teams quietly stop following it. Requiring the same light verification for both means the checkout change ships with copy-change-level proof.

The answer isn't "test everything endlessly." It's:

Ask for evidence proportional to the risk.

The useful question is not does the code look right? It's what proves this change works? Those are different questions, and the second one has a checkable answer.

For a copy change: typecheck, existing tests, a visual look. Done.

For an API-backed form change: all of that, plus browser verification of the actual flow, plus the loading, error, and success states — not just the happy path.

For checkout, auth, or anything destructive: all of that, plus end-to-end verification, assertions about application state (not just the UI), the failure paths, and regression coverage so the behavior stays guarded.

That proportionality is the whole idea behind the Frontend Change Safety Kit — a small toolkit I put together to make this repeatable. It defines three risk levels with minimum evidence bars, a per-change manifest, and instructions that coding agents can follow: collect the evidence, report it honestly, and never quietly lower the bar.

The part I like most about the kit is also the least glamorous: INSUFFICIENT EVIDENCE is a first-class outcome. If the required evidence can't be collected — no e2e runner, can't verify a browser flow, ambiguity about expected behavior — that's a failed ship decision, not a warning to wave through.

Agents are good at this, actually. They don't get tired at 5pm, and they don't rationalize that a check "probably" passes. Given explicit instructions, an agent will happily report INSUFFICIENT EVIDENCE all day — which is exactly the behavior you want guarding a checkout flow.

A green suite is an item of evidence. It is not a verdict. Verdicts come from listing the evidence the risk required and checking it off, one item at a time.