AI agent governance: the controls to define before production

Govern the job, not only the model

Agent governance begins with the business process being delegated. Name the owner, purpose, affected people, data, tools and consequences. A model risk rating without this operating context says little about what can go wrong.

Maintain a simple agent register covering production status, owner, version, permissions and review date. This creates a reliable starting point for risk, security and audit teams.

Define authority boundaries

Write down what the agent may read, draft, recommend, approve and change. Treat these as separate levels of authority. A system that can prepare a customer response does not automatically need permission to send it.

Enforce boundaries technically through narrow tools and permissions, not only prompt wording. Prompt instructions guide behaviour; access controls limit consequences.

• →Permitted data domains
• →Approved tools and actions
• →Transaction or volume limits
• →Mandatory approvals
• →Explicit prohibited outcomes

Make every run observable

Record the trigger, relevant inputs, model and prompt version, tool calls, validations, output, approval and final status. Logs should be useful to an operator without exposing unnecessary personal or secret data.

Define alert thresholds for failures, unusual costs, repeated retries and quality changes. Observability supports governance because it proves what happened and allows the team to intervene before a pattern becomes an incident.

Test quality and safety continuously

Create an evaluation set from representative work, including ordinary cases, difficult cases, missing information and attempts to redirect the agent. Score outcomes against criteria tied to the job rather than general notions of helpfulness.

Run evaluations when prompts, models, tools or connected schemas change. Sample live outputs after release and route user feedback into review. Governance is a recurring control loop, not a document signed before launch.

Prepare for incidents and retirement

Define who can pause the agent, how access is revoked, how affected outputs are identified and how work returns to a manual path. Practice this before production for consequential use cases.

Agents also need retirement criteria. Remove permissions, schedules, stored data and obsolete integrations when a job ends. An abandoned agent with active credentials is still an operational risk.