Ship AI Fixes Only After an Exploit Reproducer, Not a Model Grade
Use a trial set, an exploit reproducer, and a human review gate to decide whether an AI-generated patch ships.
The score is not the control
An AI patch can pass a model grader and still leave the exploit running. In the 1Password Off-by-1 Labs experiment, 26.0% of LLM-generated patches fully fixed the vulnerability without materially altering how the application behaved.
1Password's Off-by-1 Labs published a study examining LLM-generated fixes for new, complex vulnerabilities. The experiment created 6,080 patches using two frontier reasoning models with cyber capabilities across six recently disclosed CVEs. The hypothesis was that frontier AI models would achieve about 67% patch success rates when generating software patches. 53.9% of LLM patches were defective by leaving the bug unfixed, adding a new bug, or both.
Trail of Bits called 1Password's AI patching benchmark misleading because of flawed experimental design and released validation tools. The dispute matters because benchmark methodology decides what a score means. A score can be useful, but it cannot replace a test.
Build the gate before the model runs
A model grade is a signal. The gate makes the patch prove itself. Use this minimum set for an AI-fix pipeline.
- Define the trial set before generation. List the exact vulnerability, target version, input, and expected failure.
- Require compilation. The patch must build in the same environment as the target.
- Run the exploit reproducer. The original bug must fail after the patch.
- Check behavior. The smoke test must show no new crash, no changed API, and no new warning.
- Log the model grader. Record the score, confidence, and any disagreement with the reproducer.
- Route to human review when the grader, reproducer, or smoke test disagrees.
A vague trial lets the model solve a nearby bug, and a missing build step lets a grader accept code that cannot run. A missing reproducer lets a score stand in for proof, while a missing behavior check lets a patch break another path. A missing grader log gives the reviewer no reason to look, and a missing human gate lets automated disagreement ship.
Keep the trial set small enough to run quickly and specific enough to reject a patch that changes the wrong path. Use the same reproducer the team would use to confirm the original bug, and cover the behavior the trial expects to preserve with the smoke test.
Keep the human gate when graders disagree
Model grading can disagree with the reproducer. One model may call a patch safe while the exploit still fires. Another may flag a safe change as risky. The gate should not average those scores. Send the patch to a person when the reproducer conflicts with the model output. The disagreement is the useful signal. It shows where model confidence and reproducer results diverge.
The human reviewer does not re-run the whole test suite. The review covers the diff, the reproducer result, and the behavior log. The question is whether the patch changes the application in a way the trial did not expect. If yes, the patch stops. If no, it moves to the normal release path.
When the grader and the reproducer disagree, the reproducer wins. The grader can still be useful. It can point to a risky line, a missing edge case, or a pattern the reviewer should inspect. Treat the score as a note attached to the reproducer result, not a veto.
When the smoke test and the reproducer disagree, stop. A patch that fixes the exploit but changes behavior outside the trial is not ready. Compare the expected behavior in the trial set with the observed behavior in the smoke test. The difference is the review question.
The review gate should have a written exit condition. The patch ships when the reproducer passes, the smoke test passes, and the reviewer signs off on the diff. The patch stops when any one of those conditions fails.
The operator needs to know which trials count, which reproducer is final, and which disagreement triggers review. An unclear answer means the pipeline is not ready for production. A clear answer makes the model grade a secondary signal instead of the release decision.
The operator should keep the failed patches too. They show where the model's confidence was wrong and where the trial set needs a sharper input.