7 Layers of Agentic
AI Governance

Every governance architecture needs a foundation — a clear, organisation-wide articulation of what responsible AI means for this organisation, who owns it, and how it is enforced. Layer 1 is not a compliance document. It is the strategic contract that every subsequent layer depends on: without a defined mission, clear values, and assigned accountability, every technical control beneath it becomes a set of disconnected rules with no one accountable for the outcome.

The practical deliverables of Layer 1 are three: a Responsible AI Policy that defines what AI agents may and may not do in the organisation’s context; an AI Ethics Charter that articulates the values guiding AI deployment (fairness, transparency, accountability, human oversight); and a Governance RACI that assigns ownership — who approves new agent deployments, who is accountable when an agent causes harm, and who escalates to the board when the risk exceeds operational authority.

The Singapore IMDA Model AI Governance Framework for Agentic AI (January 2026) — the first comprehensive governance framework specifically for autonomous agents — requires each agent to carry a verifiable digital identity and an audit trail of which agent acted under whose authorisation. That requirement is only addressable if Layer 1 has established the accountability structure that determines whose authorisation governs which agent. Without Layer 1, the question “who approved this agent to act?” has no answer.

The EU AI Act adds a compliance dimension: Articles 9 (Risk Management) and 15 (Accuracy, Robustness, Cybersecurity) mandate conformity assessments for high-risk AI systems. A well-constructed Layer 1 Responsible AI Policy maps directly to these obligations — giving legal, compliance, and engineering teams a shared reference point rather than three separate interpretations of the regulation.

The most common governance failure is not inadequate policy — it is inadequate visibility. 68% of employees already use AI tools without IT approval, creating a Shadow AI visibility gap that expands the attack surface faster than security teams can map it. Before any other governance layer can function, an organisation must maintain a complete, current inventory of every AI agent, model, tool, and integration operating within its environment — both sanctioned and unsanctioned.

The AI Agent Registry is Layer 2’s primary artefact. For each registered agent, it captures: agent identity and purpose; the capabilities and permissions it holds; the data sources it can access; the tools it can invoke; its owner and approver; the third-party integrations it depends on; and its last audit date. Agentic AI introduces identity risks that traditional IAM cannot address. Privilege drift, shadow agents, MCP bypass, and broken delegation chains all stem from treating autonomous agents like static human users or simple service accounts. The registry is the antidote — it makes every agent a first-class, named, accountable entity.

Transparency extends to third-party integrations. When an agent calls an external LLM API, uses an MCP server hosted by a vendor, or delegates a subtask to a third-party agent service, governance extends to that boundary. When your agents invoke external AI services — third-party models, SaaS APIs — governance extends to the boundary. Layer 2’s registry must track these dependencies explicitly, enabling the organisation to understand what external systems have access to its data and can take actions on its behalf.

Automated discovery tooling — scanning code repositories, API logs, cloud IAM records, and network traffic — is essential at any meaningful scale. An organisation relying solely on voluntary agent registration will not discover its shadow AI estate; automated scanning will.

Not all AI agents carry the same risk. An agent that summarises internal meeting notes poses fundamentally different risks from one that approves financial transactions or modifies customer records. Risk classification is the mechanism that translates that difference into proportionate governance — ensuring that high-risk agents receive high-impact controls while low-risk agents are not burdened with disproportionate overhead.

Layer 3 requires a formal risk assessment for every agent in the Layer 2 registry. The assessment dimensions align with the NIST AI RMF’s Map and Measure functions: impact (what is the worst credible outcome if this agent acts incorrectly?), likelihood (how likely is that outcome given the agent’s design?), reversibility (can the damage be undone?), and regulatory exposure (does this agent fall under the EU AI Act’s high-risk categories, HIPAA, GDPR, or financial services regulation?).

The EU AI Act mandates conformity assessments for high-risk systems under Articles 9 and 15. The Singapore Model AI Governance Framework requires risk-proportional oversight, which the autonomy-level semantics mirror. A four-tier classification system — Critical, High, Medium, Low — maps assessment scores to proportionate governance requirements: Critical agents require human approval for every consequential action; High agents require HITL at defined checkpoints; Medium agents run with monitoring; Low agents run with logging only.

Risk assessment is not a one-time pre-deployment exercise. Risks such as tool misuse, excessive permissions, and goal hijacking show why controls need to work while an agent is running, not just before deployment. Trigger re-assessment whenever an agent’s capabilities, data access, or operating environment changes materially — or on a fixed cadence (quarterly for Critical and High, annually for Medium and Low).

Layer 4 is where governance becomes engineering. The controls and guardrails established at this layer are the technical mechanisms that enforce the policies defined in Layer 1, scoped to the risk classifications from Layer 3. They are not optional — they are the difference between a governance framework that works and one that looks good in a slide deck.

The control stack has three levels. Access controls and permissions: every agent operates under a least-privilege identity with the minimum permissions necessary for its defined task. Permissions are scoped, time-limited, and revocable. Privilege drift — where agents accumulate permissions that exceed what they need for any single task — is one of the most common agentic risk patterns. Role-based and attribute-based access controls, enforced programmatically through IAM systems and MCP gateway policies, prevent this accumulation from occurring silently.

Behavioural constraints and safety guardrails include: input validation (blocking prompt injection and malformed tool calls before they reach the model); output filtering (checking model responses against content policies and data leakage rules before delivery); scope enforcement (rejecting actions outside the agent’s defined task scope); and hard stops (conditions under which the agent must halt and escalate, regardless of the model’s confidence). Every agent needs a clear escalation path — conditions under which it must stop autonomous operation and involve a human. The agent pauses its workflow, packages the current context, and routes to the designated human approver. The workflow resumes only upon human authorization.

Output validation and human override close the loop. For Critical and High-risk agents, every consequential action — irreversible operations, significant financial transactions, communications to external parties — requires a human approval gate before execution. This is not advisory; it is a hard enforcement point in the system’s architecture. The AI agent literally cannot proceed without the approval token. This is the EU AI Act’s Article 14 (Human Oversight) obligation operationalised as code.

Controls govern what agents may do. Monitoring reveals what they actually do. The gap between the two is where risk lives. Layer 5 establishes the continuous observability infrastructure that closes this gap — giving governance teams the real-time visibility they need to detect anomalies, policy violations, and behavioural drift before they compound into incidents.

The five observability pillars for agentic systems mirror those for data systems but with critical extensions. Activity logging: every agent action — every tool call, every API invocation, every document read, every output generated — is logged with timestamp, agent identity, authorising context, and outcome. Logs must be tamper-evident and retained for the regulatory retention period. Decision tracing: for consequential decisions, the reasoning chain that led to an action must be captured, not just the action itself — this is the foundation of explainability required by the EU AI Act. Anomaly detection: statistical baselines per agent enable automated alerting when behaviour deviates — an agent suddenly calling tools it has never used, accessing data outside its normal scope, or generating outputs dramatically longer than its baseline.

Performance monitoring tracks accuracy, latency, and task completion rates over time — enabling teams to detect model drift (where a model’s performance on its task degrades without any code change) and capability regression (where a new model version underperforms the previous one on production tasks). Policy violation detection runs continuously: every agent action is checked against the rules established in Layer 4, with real-time alerting when violations occur rather than batch retrospective review.

The NIST AI Agent Standards Initiative (February 2026) and the associated NCCoE concept paper on software and AI agent identity and authorisation explicitly require continuous monitoring as a governance function. NIST AI 100-2 (March 2025) explicitly names AI agents as a threat surface for the first time, and continuous activity monitoring is the primary detection mechanism for agentic threat patterns.

Governance without evidence is assertion. Layer 6 generates the evidence base — the documented, independently verifiable proof that agents operate as governed. It is the layer that satisfies the auditor, the regulator, and the board. Without it, Layers 1–5 are well-intentioned but legally indefensible.

The assurance programme has four components. Regular audits review agent configurations, permission scopes, activity logs, and incident records against the policies established in Layer 1. For Critical and High-risk agents, audits should occur quarterly; Medium annually. Audit findings are documented with remediation timelines and sign-off from the agent’s designated owner. Red-teaming and adversarial testing subject agents to scenarios designed to find governance failures: prompt injection attempts, permission escalation attacks, goal hijacking, and out-of-scope instruction following. Test how the agent handles unexpected instructions, restricted data, failed tools, and requests outside its role. This helps uncover issues before they reach production.

Stress testing and edge case evaluation test agent behaviour under unusual conditions: high-volume loads, tool failures, ambiguous instructions, conflicting constraints, and adversarial inputs. An agent that behaves correctly in normal conditions but fails dangerously under stress is not governed — it is lucky. Compliance mapping ensures that audit evidence is structured to satisfy regulatory requirements: EU AI Act Articles 9, 14, and 15; ISO/IEC 42001:2023 AI management system requirements; NIST AI RMF Govern and Manage functions; and any sector-specific regulation (HIPAA, DORA, FCA guidelines).

OWASP’s Top 10 for Agentic Applications (December 2025) provides the most comprehensive current taxonomy of agentic-specific vulnerabilities — including prompt injection, insecure tool execution, excessive agency, and data exfiltration through tool calls. Layer 6 testing programmes should cover all ten categories for Critical and High-risk agents.

Governance that stops at deployment is not governance — it is a pre-launch checklist. Layer 7 completes the loop by establishing the response and learning mechanisms that make the entire governance framework adaptive. Every incident, every near-miss, every audit finding, and every new regulatory requirement is an input to a structured improvement cycle.

The incident response capability has four components. Detection and containment: when Layer 5 monitoring flags an anomaly or violation, the response playbook triggers immediately — the agent is paused if necessary, the scope of potential impact is assessed, and the designated escalation path from Layer 1 is activated. Speed matters: the difference between a contained incident and a significant breach is often measured in minutes. Rollback mechanisms: every agent deployment must have a documented and tested rollback procedure — a path back to the previous version or to a safe stopped state — that can be executed without the agent’s cooperation. An agent cannot be given the keys to its own kill switch.

Post-incident review is mandatory for every incident above a defined severity threshold. The review addresses root cause (not just immediate cause), contributing factors, detection latency, response effectiveness, and preventive controls that would have caught the issue earlier. Findings are formally logged and tracked through to remediation. Policy and control updates flow from the post-incident review: if an incident reveals a gap in Layer 4 controls, Layer 3 risk classifications, or Layer 2 registry completeness, those layers are updated before the agent returns to operation.

Continuous improvement extends beyond incident response. The threat landscape for agentic AI is evolving faster than any other technology domain. New attack patterns (MCP bypass, cross-agent prompt injection, authorisation chain attacks), new regulatory requirements (AI Act delegated acts, NIST IR 8596), and new capabilities (more powerful models with broader tool access) all require the governance framework to evolve continuously. A governance board review cadence — quarterly for the framework, annually for the full policy suite — ensures that the framework remains current rather than becoming a historical artefact.