Local agent harnesses need egress controls outside the model: deny public IPs, allowlist fetch methods, and fail hard on mock errors.
Pin the fixed pydantic-ai release and run three exposure checks before an agent fetches untrusted pages.
If generated code can inherit production environment variables, the sandbox is decorative; verify the execution path and rotate exposed credentials.
There's one thing an RTA user at the table can never do, and a player who trains against the engine does every day. It's the difference between winning a hand and owning the game.
A GET-only agent can still run code if egress allows echo, screenshot, and shortener services to assemble and return payloads.
Run a pre-buy check: public benchmark score, per-task cost, and fleet pricing assumptions before you deploy a computer-use agent.
Check dataset version, run date, leak-channel closure, pass-to-fail flip count, and pre-audit flag before using a SWE-Bench Pro score for coding agent model selection.
Treat agent completion as an artifact check: audit which files were opened and run planted-defect tests before accepting a review.
Before trusting an AI employee vendor, map approval points, verify logged actions, and test agent handoffs.
Name every agent, owner, and data path, then test whether you can stop a runaway agent before it becomes an incident.
A runtime monitor that only reads command text will miss encoded, split, and retried actions; audit it as a layer, not the whole stack.
Run a pre-review agent audit, count only generally available controls, and treat missing task-scoped grants and delegation tracing as gaps to close.
A control plane must prove identity, audit, data, approval, and cost controls before you route agent traffic through it.
Agent access mapping is funded, and operators can still run a practical audit this week.
A weekly capacity triage for operators: read usage, map workflows, and move non-real-time work before the ceiling breaks.
A frontier-agent claim is only as strong as the harness, runner, and re-audit behind it. Use this one-page check and five-job internal eval before deploying.
The Fable 5.1 cache-read cut is a decision trigger, not a blanket discount; measure cache-hit share before migrating.
Use a trial set, an exploit reproducer, and a human review gate to decide whether an AI-generated patch ships.
Check unattended task boundary, decision telemetry, and cost-per-task math before putting Devin on production work.
Treat incoming repositories as hostile intake: disable executable Git config paths and pin patched agent versions before any index-refreshing command runs.
Treat self-hosted agent orchestration platforms as production credential vaults, then run a fast breach triage after an exploited flaw.
Treat every agent as a low-trust non-human identity: inventory it, scope it, expire it, revoke it, and keep tokens out of logs and builds.
Treat every coding-agent plugin as self-updating code, verify the resolved commit, and lock the update path before it runs with your credentials.
Verify autonomy, model routing, review gates, failure rates, rollback paths, and logs before giving a coding agent access to production repos.
The Last Honest Edge in Poker: Training Against the Machine Check benchmark, cost, and fleet math before buying a robosecretary Audit agent vendor end-to-end claims with three checks
Three Egress Checks to Stop Your Local Agent From Hitting Live Sites Audit Agent Code Execution: Prove Sandbox, Rotate Secrets Run a Claude Code capacity check before the 125-unit weekly limit