ANTANOX — AI Agent Security & Reliability

2026-09-27 · VERIFICATION

FIVE TO SEVEN DAYS BECAME THREE. THE REVIEW IS THE PART THAT GREW.

What agent-written code does to a one-person engineering practice: the numbers I have, the four failure modes I keep hitting, and what I cannot prove yet.

The number

A single feature, frontend and backend together, used to take me five to seven days. Now it takes two days to build, plus a day — sometimes a bit more — to review. Three to three and a half days end to end. That is a third to more than half of the calendar time gone, and the ratio holds for most tasks.

One exception, and it is the exception that matters: as the code grows and gets more complex, the review side grows with it. The build stays fast. The review does not.

The trend I keep running into

Most of the time people get back is being spent on shipping faster, not on reviewing more carefully. The working assumption is “if it works, then it is fine.” I understand the pull — I feel it too. But it moves the entire cost of the speedup onto the one step that was keeping it honest, and I do not think that trade has been priced correctly.

What I actually run

Review is allocated by risk. Core logic gets read by a human; everything else passes through explicit, versioned quality rules and a loop of reviewer agents that keeps filtering until nothing potentially harmful is left. The reviewer agents are a filter, never the approver. Sensitive paths and high-blast-radius areas run on self-hosted models, with a human in the loop at named checkpoints — a deliberate decision per area, not a default.

Four things agent-written code does wrong — none of which fail the tests

  1. Drift. It builds features that were never asked for. Small, plausible, tangential features that look like generosity.
  2. Local decisions that contradict the future. The approach is fine for the thing in front of it and wrong for the third thing after it. Nothing in the current task can tell you that.
  3. The confident shortcut. Given an easy path and a reliable one, it takes the easy one — and presents both with identical confidence. The complexity it avoided lands on you later, with interest.
  4. It all passes. This is the actual problem. Tests verify that the code does what it says it does. They do not verify that it is the right thing to have built. A drifting, over-engineered, future-hostile implementation can be fully green.

So “if it works, then it is fine” is not a lower standard. It is a standard that does not measure the thing most likely to hurt you. Green is not the same as correct, and it was never meant to be.

What catches what

The reviewer-agent loop catches pattern-level problems: sloppiness, obvious hazards, missing cases. It has never caught drift, because drift is not a defect in the code — it is a divergence from intent, and intent lives in my head, not in the repository. The only thing that has ever caught an unrequested feature is a human who knew what was asked for, reading the diff.

That is why the review time went up when I started using agents, and why it keeps going up as a project grows. It is not friction I have failed to engineer away. It is the part of the process that has to stay slow, because it is the only part that can see the failure the tests cannot.

What I cannot prove yet

I have one real metric — the cycle time above — and four failure modes I have watched happen. I do not have a defect-escape rate, and I am not going to invent one: I have not instrumented for it. That is the next thing I build. Anyone quoting a clean escape rate for AI-written code is measuring something narrower than the name suggests.

The ledger, such as it is

  • Feature cycle time: 5–7 days → 3–3.5 days
  • Review as a share of the cycle: roughly a third, and growing with complexity
  • Build-to-review ratio now: about 2:1, tightening as the codebase grows
  • Defect escape rate: not measured yet
  • Failures caught by the reviewer-agent loop: pattern-level only
  • Failures caught only by a human who knew the intent: the drift cases

What I would never do again

  • Treat “it works” as done. Passing tests are the start of review, not the end of it.
  • Let review be the variable that absorbs schedule pressure. It is the only step that can see intent drift, and it is the first thing every deadline reaches for.
  • Accept a shortcut without asking what it forecloses. The easy path is usually correct for today and expensive for the third feature after it.

The bar

I do not hold agent-written code to a higher standard than my own. Humans do not write consistent, completely safe code either, and pretending otherwise is not a standard — it is a way to lose an argument. The same bar applies to both. What changed is the volume, and volume is a capacity problem, not a quality problem.

Which is why the honest summary is this: the speed is real, and so is the fragility. The difference between the two is how much review you are willing to pay for.

If you are staring at the same trade in a codebase you did not write: the 10-Day Architecture Audit.