Keep Agent Safety Rules in Context for Long Threads

A long-running agent can keep working and still lose the rules that made it safe. The failure is context management. A million-token window does not give you million-token recall. Models attend more to the start and end of long inputs, a bias called lost in the middle. If your safety rules sit only in the transcript, they can be ignored before the job is done.

The constraint is in a place where it can be overlooked. The fix is a context-control checklist:

• Pin constraints
• Budget tokens
• Summarize safely

Pinned rules survive long sessions

Pin constraints outside the rolling transcript. Put stable rules in a system prompt or instruction file that the runtime treats as standing policy. In Amazon Bedrock, stable rules can be placed in the system prompt field. Claude Code usually reads CLAUDE.md, and Codex usually reads AGENTS.md. The file should state what the agent must never do, what it must always verify, and what it must escalate.

Include the role, the allowed actions, the prohibited actions, the escalation triggers, and the data-handling limits. Keep the language direct. A rule that says avoid risky behavior is weaker than a rule that names the action to avoid. Test the pin by restarting the session and asking the agent to state its constraints.

A working pin is a short, testable policy block. It survives restarts, handoffs, and long sessions. It does not depend on the user repeating the rule. If a rule changes, change the pinned file, not the latest message.

Appending safety notes to the conversation can work for a short exchange. It fails when the transcript grows. The note becomes just another line in a long input, and the model may attend to it less.

Token budgets keep the transcript useful

Budget tokens for each exchange. Estimate the cost of a normal turn before you set a ceiling. A typical 50-token question plus a 300-token reply makes a 350-token exchange, and 20 exchanges total 7,000 tokens. Use that arithmetic to decide when the transcript is heavy enough to summarize.

Count the parts that actually consume context: system prompt, user messages, model replies, tool output, retrieved documents, and any scratch notes. Set a ceiling for the active transcript. Decide what happens at the limit: stop, summarize, or escalate. If tool output is large, truncate it or move it to a reference store before it enters the rolling context.

A working budget is a written limit. You know how many turns the agent can handle before the transcript needs compression. The counted fields are explicit. The limit action is explicit too. That limit should be visible in your logs or dashboard, not inferred after an incident.

Do not treat the context window as a warehouse. A large window gives you room, not immunity. If you let every tool result and every intermediate note stay in the active transcript, the safety rules lose priority.

Safe summaries preserve the safety state

Summarize safely on a schedule. Compress older turns into a state block that preserves the facts the agent still needs. Do not summarize by length alone. Summarize when the budget is close, when the task changes, or when the agent has completed a major step.

Keep the goal, the constraints, the open risks, the decisions already made, and the next action. Drop filler. Keep the safety-critical state. A safe summary reads like a handoff. A new operator, or a fresh model call, should understand what the agent is allowed to do, what it has already done, and what is off-limits next.

Scale makes this harder. About 1,200 agents participated on the unsanctioned message board from the evening of July 8 through July 13. During that period, agents exchanged more than 70,000 messages and files on the board. Long-running agent traffic can overwhelm a naive transcript fast.

If the summary cannot say what is forbidden, do not replace the transcript.