Turn 2,000 Agent PRs Into a Shippable Fork With 5 Gates

DoltLite reached Beta version 0.50.0 five months after launch. The operator should cap review volume and enforce gates. Agent-driven development can produce a lot of code. It can also produce a lot of noise. The difference is control. A large fork can move fast when the merge path is narrow. The operator’s job is to make the narrow path enforceable.

A fork inherits a known surface. Agents can work against a hard target. The target is not better code. The target is continued compatibility with the underlying system, with exceptions.

Why volume is not the failure mode

High PR volume is not the problem. Untracked divergence is the problem. An agent can open a clean patch. It can also open a patch that changes behavior in a way no human will notice. The review gate must catch the second kind before it becomes the default.

The strongest signal in the project is the test surface. DoltLite passes all 5.8M sqllogictest queries. That says the core query path still behaves. The second signal is that the remaining gaps are counted, visible, and owned.

The five gates

Use the gates as a merge policy, not a checklist you run once. Each gate should block work until the condition is met. The goal is to make agent PRs safe to review at scale.

1. Cap PRs. Limit agent PRs entering review per cycle. Base the cap on human review capacity. Stop generation if the queue grows. Fix the pipeline first.
2. Run the compatibility suite. Require a hard test suite against the base system. It must run on every PR and catch regressions in common paths. A passing suite is the minimum condition for review.
3. Update the divergence ledger. Track every known difference from the base system. Each entry needs an owner, a reason, and a status. The ledger makes exceptions visible and gives reviewers a map of allowed differences.
4. Trigger human review. Define changes that require human review before merge. Include storage changes, public API changes, error behavior, performance-sensitive paths, and new divergences. Agents can prepare the review. Humans decide whether the risk is acceptable.
5. Require a migration path. If the fork changes a format, protocol, or public contract, require a migration plan before the change lands. The plan must explain how existing users move forward.

The storage format is stable for Beta, with a supported migration path for future breaking changes. It tells users the fork is not a moving target. It also tells the team that future changes need a plan before they happen.

How to run the gates in practice

Start with the merge path. Make the gates part of the branch protection rules. A PR should not be mergeable until the required checks pass. The checks should include the compatibility suite, the divergence ledger update, and the human review trigger. If a change is small, the gates should still apply. Small changes are where silent drift starts.

Keep the divergence ledger visible. Put it in the repository, not in a side document. Reviewers should be able to see the current exceptions while reading a patch. If a PR adds a new divergence, the ledger entry should be part of the same change. That makes the exception explicit. It also makes the review faster, because the reviewer is not hunting for hidden behavior changes.

Use the PR volume cap as a feedback signal. If agents are producing more PRs than the team can review, the system is already broken. The fix is not to review faster. The fix is to reduce the number of changes that need review. Consolidate small patches. Reject speculative work. Require a clear problem statement before an agent opens a PR. The cap protects the review process from becoming a bottleneck.

Make code review a decision, not a formality. The reviewer should answer three questions: Does the change preserve expected behavior? If not, is the divergence documented? If the change touches a protected surface, is the migration path complete? If the answer to any question is no, the PR stops. That is the whole point of the gate.

What to do next

If you are shipping or evaluating agent-built open source, start with the compatibility suite. Without a hard suite, the other gates are weak. The suite is the shared language between agents and humans. It tells the team what “works” means before anyone argues about a patch.

Then build the divergence ledger. It is the cheapest way to make a fork auditable. A ledger does not prevent divergence. It makes divergence visible. It also gives you a basis for deciding which differences are acceptable and which ones need to be closed.

Finally, set the human review trigger early. Do not wait until the codebase is large. Define the protected surfaces now. Storage format, public API, error handling, and performance-critical paths are good starting points. The trigger should be simple enough that agents can classify their own work. It should also be strict enough that risky changes cannot slip through.

The example shows that agent-driven development can produce a shippable result when the constraints are explicit. The operator’s job is not to trust the agents. The operator’s job is to build a path where the agents can only move forward when the system says it is safe.