Running AI Agents in Production Without Losing Control

Make the loop explicit

An agent alternates between model decisions and tool results. Persist the accepted input, decision, selected tool, arguments, result reference, and next state. Then a crash restarts from the last boundary instead of asking the model to rediscover what happened.

Keep transient token streaming separate from durable state. A partial sentence is presentation; an approved tool request is a business transition.

Put hard controls outside the prompt

• Allow-list tools and validate arguments against schemas.
• Set per-run token, money, time, and step budgets.
• Require approval for payments, sends, deletion, or publishing.
• Use stable intent keys for tool calls that change external state.
• Redact secrets and sensitive data from model context and logs.
• Stop on repeated failures instead of extending the loop indefinitely.

Operate the agent as a workflow

Measure task success, human override rate, tool error rate, unknown effects, cost per completed outcome, and time spent waiting for approval. Sample traces for qualitative review, but keep a structured audit record for every consequential action.

Orch8 can hold durable agent state, approval waits, and external worker tasks. Model quality and tool authorization remain application responsibilities. Start with one narrow outcome whose failure is easy to detect and reverse.

How should an agent run become a state machine?

Represent the run as explicit durable stages: accept objective, assemble bounded context, propose an action, validate policy, request approval when required, dispatch one tool call, record the result, evaluate completion, and either continue or stop. Persist the model, prompt or template version, tool schema version, budgets, accepted messages, action intent, and result digest at each boundary. A transcript alone does not say which action was authorized or safe to repeat.

Keep reasoning text separate from control state. The engine should decide whether a run may advance from validated to dispatched based on structured fields and policy, not by parsing a model's prose. Treat model output as untrusted input: validate the schema, allowed tool, arguments, resource scope, and budget before creating an effect intent.

Give every proposed action a stable ID derived from the run and decision boundary. A retry of the same approved action reuses that ID; a revised proposal receives a new one. This lets workers recover from timeouts without converting a network attempt into a second email, payment, deletion, or deployment.

How do you constrain tools and permissions?

Expose narrow capabilities rather than a general shell or unrestricted API client. Define typed arguments, allowed resource prefixes, maximum batch size, timeouts, and whether the call is read-only, reversible, or consequential. Resolve authorization from the user and tenant context at dispatch time; never trust a role or account identifier supplied by the model.

Place irreversible or high-impact actions behind deterministic policy and human approval. The approval view should show the exact action, target, diff or payload summary, cost, permissions, and expiration. If arguments change after approval, invalidate it. Record approver identity and policy version, then bind the approval to the action digest.

Sandbox data transformation and apply output limits. Defend against prompt injection in retrieved pages by treating retrieved instructions as data, restricting available tools, and requiring policy checks outside the model. Log denied actions as security signals, but redact sensitive content before observability export.

• Use least-privilege, tenant-scoped credentials.
• Validate tool name and arguments against a versioned schema.
• Bind approvals to an immutable action digest.
• Cap iterations, tokens, wall time, tool calls, and spend.
• Require explicit policy for network, write, delete, payment, and deploy actions.
• Provide a kill switch that stops new dispatch while preserving evidence.

What does safe retry and recovery look like?

Classify failures by boundary. A model timeout before an accepted proposal can be regenerated under the run budget. A validation rejection should return structured feedback or stop, not retry unchanged. A tool timeout after dispatch is an unknown outcome: query the tool or provider by action ID before deciding whether another call is safe.

Store checkpoints after accepted boundaries and resume from the latest compatible state. Do not replay completed tools merely to reconstruct context; load their retained result or a durable summary. If a prompt or tool schema changed, route the run to a compatible worker, migrate with validation, or require a new decision. Silent reinterpretation makes the audit trail unreliable.

Operators need guarded controls to pause, cancel, approve, deny, reconcile, and terminate. Cancellation should stop future dispatch and attempt cooperative interruption, while acknowledging that an already accepted external action may finish. Compensation is a separate authorized action linked to the original receipt.

if action.status == confirmed: reuse(result)
if action.status == rejected: stop_or_replan()
if action.status == unknown: reconcile(action_id)
if action.status == prepared: dispatch_once(action_id)

Which production evaluations and metrics matter?

Evaluate correctness at the action boundary, not only answer quality. Build scenarios for prompt injection, malformed tool arguments, permission escalation, duplicate delivery, ambiguous tool response, approval expiry, budget exhaustion, model change, and worker termination. Assert allowed actions, denied actions, external request count, durable state, and user-visible result.

Monitor completion by task class, policy rejection, approval rate and latency, tool error and unknown-outcome age, iterations per run, token and monetary cost, repeated-action conflicts, human override, and unsafe-action escape rate. Segment by model, prompt version, tool version, and release. Sampled transcripts help diagnosis but do not replace structured safety metrics.

Roll out model or prompt changes as versioned releases with a fixed evaluation set and a production canary. Compare quality, cost, latency, tool selection, and policy outcomes. Retain the ability to stop new runs while compatible workers finish or safely checkpoint active ones.

How should an agent incident be investigated?

Freeze new high-risk dispatch without deleting the affected runs. Preserve accepted messages, prompt and model versions, retrieved source references, policy decisions, approval digests, tool schemas, action intents, provider receipts, and operator actions. Determine the last trustworthy boundary before inspecting free-form transcript explanations.

Classify whether the failure came from context, model output, schema validation, authorization, policy, approval, dispatch, provider ambiguity, or result interpretation. Count every external request and compare it with durable action identities. The remediation should strengthen the failed boundary with a rule, narrower tool, safer default, or regression scenario—not depend only on asking the model more emphatically.

Before resuming, replay the case in an isolated environment and test adjacent variants. Document affected scope, user impact, evidence limitations, recovery decisions, and release versions. Retain a redacted fixture in the evaluation suite so future model and prompt changes cannot quietly reintroduce the behavior.

• Preserve state before operator mutation.
• Separate model error from control-system error.
• Reconcile consequential tools with provider truth.
• Turn the failure into a repeatable evaluation.