Call-bot.ai

AI agents & agentic automation12 min read

AI Agent Governance and Security: A Checklist for Businesses

An AI agent that can read your systems and act on them has a larger attack surface than a chatbot that only answers. This checklist turns OWASP, NIST, the EU AI Act and May 2026 joint government guidance into controls you can verify: what to restrict, what to log, who owns the agent and how to stop it.

Operators seated at their screens, facing a large supervision display in a control room

Photo: Hyundai Motor Group · Pexels

An AI agent combines a language model, tools that act on real systems and the credentials to reach them, which is what separates it from conversational AI that only answers. The model can be steered by any text it reads: an email, a web page, a support ticket. In December 2025, the UK's National Cyber Security Centre (NCSC) wrote that prompt injection is unlikely ever to be properly mitigated the way SQL injection was, and that the realistic goal is to reduce likelihood and impact. OWASP's 2025 list said it was unclear whether fool-proof prevention exists, and its 2026 list (see below) says no reliable prevention mechanism exists today. The practical rule that follows: assume the model can be tricked or can simply be wrong, and limit what a tricked or mistaken agent can do.

This page is a checklist for AI agent governance and security, not a framework. It is a selection, not an exhaustive list: it does not cover data protection law such as the GDPR or sector-specific rules, and completing it does not by itself guarantee security or legal compliance. NIST's AI Risk Management Framework states that its actions are not a checklist, so we cite its subcategories only where they support a control. OWASP numbers below are those of its 2025 list (LLM01:2025, LLM06:2025, LLM10:2025). OWASP has since published a 2026 list (resource page dated August 3, 2026) that renumbers them: Prompt Injection remains LLM01, Excessive Agency becomes LLM03 and Unbounded Consumption becomes LLM06. If your first agent faces customers, read customer service AI agents: examples, ROI and risks as well.

Agentic AI security: where an AI agent can go wrong

Prompt injection (OWASP LLM01:2025). OWASP describes inputs that alter a model's behavior in unintended ways, either from a user or through external content such as websites and files, and says retrieval-augmented generation and fine-tuning do not fully mitigate it. Its first attack scenario is a customer support chatbot told to ignore its guidelines, query private data stores and send emails. EchoLeak (CVE-2025-32711), analyzed in a September 2025 arXiv paper, was a zero-click prompt injection in Microsoft 365 Copilot that allowed data exfiltration through a single crafted email.

Excessive agency (LLM06:2025). OWASP defines it as the vulnerability that lets damaging actions follow unexpected, ambiguous or manipulated model outputs, whatever the cause, and traces it to excessive functionality, permissions or autonomy. Its example: a product-recommendation agent needs read access to a products table, not rights to change or delete records.

Agentic AI risks. OWASP's Top 10 for Agentic Applications (December 9, 2025) lists ASI01 Agent Goal Hijack, ASI02 Tool Misuse and Exploitation, ASI03 Identity and Privilege Abuse and seven more, including memory poisoning, insecure inter-agent communication and rogue agents. The joint guidance "Careful adoption of agentic AI services" (May 1, 2026; CISA, NSA and the cyber agencies of Australia, Canada, New Zealand and the UK) groups risks into privilege, design and configuration, behavior, structural and accountability. One scenario: a support-ticket triage agent whose unvetted scheduling component is compromised, giving the attacker a path to the agents handling billing and refunds.

AI agent security controls: restrict, approve, log, stop

Least privilege. Give each agent the narrowest tools and permissions its task needs. The joint guidance calls strict adherence to least privilege critical; OWASP prefers narrow tools to open-ended ones such as a shell or arbitrary URL fetching.

Scoped, short-lived credentials. Each agent gets its own identity, not a shared admin key. The NCSC advises temporary credentials and revoking elevated access when a task ends; the joint guidance recommends just-in-time credentials for high-impact actions.

Authorization outside the model. OWASP calls it complete mediation: enforce permissions in the downstream system rather than relying on the model to decide what is allowed. The NCSC favors deterministic safeguards that constrain what the system can do over trying to keep every malicious input from the model, and adds that if a system cannot tolerate the remaining risk, it may not suit a language model.

Human approval for sensitive actions. Decide in advance which actions need sign-off. The joint guidance suggests classifying actions by impact, likelihood and reversibility, and says system designers or operators, not the agent, should decide when approval is required.

Spend and rate limits. OWASP LLM10:2025 (Unbounded Consumption) covers "denial of wallet", where attackers exploit per-use pricing, and lists rate limits, quotas and timeouts. OWASP adds that rate limiting and monitoring do not prevent excessive agency but limit the damage.

Logging and traceability. The joint guidance recommends monitoring all agent operations, including internal steps and tool calls, not only inputs and outputs.

Sandboxing and isolation. Segment environments to limit the blast radius if one agent fails or is compromised, and test in a sandbox before production (joint guidance).

Kill switch. NIST AI RMF MANAGE 2.4 expects mechanisms to supersede, disengage or deactivate AI systems whose performance is inconsistent with intended use. Name who can trigger it, and rehearse it.

Guardrails. System-prompt rules and filters help, and OWASP lists them among mitigations, but they are one layer, not a boundary. The joint guidance puts hard constraints, such as deny lists and API-level safety policies, at the deployment layer. Enforce AI agent guardrails in code, not only in prompts.

Agentic AI governance: who owns what

Ownership. The NCSC says to be clear about who owns an agentic system, who approves its access, who monitors it, who reviews incidents and who can stop it, before it touches real systems. NIST GOVERN 2.1 asks for documented roles; the joint guidance says to define legal accountability and risk ownership in policy.

Agent inventory. NIST GOVERN 1.6 calls for mechanisms to inventory AI systems, and the joint guidance recommends a trusted registry of agent identities, reconciled periodically against the agents actually running. One row per agent: owner, purpose, tools, permissions, data touched, approval rules, last review.

Pre-production evaluation. NIST MEASURE 2.7 expects security and resilience to be evaluated and documented. The joint guidance recommends threat modeling, red teaming in a sandbox and phased deployment starting with low-risk, well-defined tasks. For how we evaluate voice agents, see our testing methodology.

Periodic review. ISO/IEC 42001 is a management-system standard: requirements for establishing, implementing, maintaining and continually improving an AI management system. In practice, schedule reviews of permissions, logs and incidents, and re-evaluate whenever the model, a prompt, a tool or a data source changes.

Incident process. The NCSC says to plan for agent failures, misuse and loss of control; NIST MANAGE 4.3 covers communicating incidents and errors to relevant parties. Define what triggers the kill switch and who is called.

What NIST and ISO say

NIST. AI RMF 1.0 (released January 26, 2023) is voluntary and organized around Govern, Map, Measure and Manage; NIST says a revised version is in progress. The Generative AI Profile (NIST AI 600-1, July 26, 2024) lists twelve risks, including confabulation, information security and human-AI configuration.

ISO/IEC 42001:2023 (December 2023) is designed, per ISO, for organizations providing or using AI-based products or services. Nothing we read makes it mandatory.

What the EU AI Act requires, and from when

The AI Act is Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744 (the "Digital Omnibus on AI", which the Commission also calls the "AI Omnibus"; published July 24, 2026 and in force July 27, 2026). Per the consolidated text:

  • AI literacy (Article 4) has applied since February 2, 2025. As amended, providers and deployers must take measures to support their staff's AI literacy; the duty does not require any specific level for any individual.
  • Transparency (Article 50) applies from August 2, 2026. Providers must design AI systems that interact directly with people so users are told they are dealing with an AI, unless obvious, at the latest at first interaction. Providers of AI systems that generate synthetic content and were placed on the market before August 2, 2026 have until December 2, 2026 to mark outputs (Articles 50(2) and 111(4)).
  • High-risk systems: the oversight requirements (Article 14, including a "stop" button or similar procedure) and deployer duties (Article 26: assign trained overseers, monitor, suspend use on risk, keep the automatically generated logs under their control for at least six months, unless other EU or national law, in particular data protection law, provides otherwise) apply from December 2, 2027 for Annex III uses and August 2, 2028 for products under Annex I. Whether an agent is high-risk depends on its use; Annex III lists eight areas, including employment and workers' management (point 4) and access to essential private and public services (point 5), which covers uses such as evaluating creditworthiness and classifying emergency calls.

This is a summary of selected provisions, not legal advice and not the Act's full set of obligations; check how your use case is classified.

The checklist

Control Why How to verify
Agent inventory, named owner Unowned agents cannot be reviewed or stopped (NIST GOVERN 1.6; NCSC) Every production agent is registered; reconcile quarterly against what runs
Least-privilege tools Excessive agency (OWASP LLM06:2025) Compare each tool's permissions with the task; remove unused write and delete rights
Per-agent, short-lived credentials A stolen or shared key lets an attacker act as the agent Audit: no shared admin keys, expiry set, revocation tested
Authorization outside the model The model can be manipulated (OWASP LLM01:2025; NCSC) In a test, try to make the agent do a forbidden action; the downstream system must refuse
Human approval for sensitive actions Limits high-impact, hard-to-reverse errors Written list of approval-required actions; test for bypass; sample the approval log
Spend and rate limits Runaway cost (OWASP LLM10:2025); damage limitation (OWASP LLM06:2025) Budget alerts and hourly caps; simulate a loop and confirm it stops
Full logging of prompts, tool calls, outputs Incident reconstruction; log retention for high-risk uses Trace one recent action from user input to system change
Isolation and sandbox Limits the blast radius Test environment separate from production; no agent can call another's tools
Kill switch NIST MANAGE 2.4; Article 14(4)(e) for high-risk systems Timed drill: how long to disable the agent and revoke its credentials
Adversarial pre-production tests Prompt injection may not be fully preventable Injected text in emails, tickets and pages; repeated runs; results documented
Disclosure and a route to a person Article 50(1) from August 2, 2026 (a provider duty); customer trust Check the first message and the path to a human on each channel
Incident and review process Failures will occur Playbook with owner and contacts; dated review records

Frequently asked questions

What is AI agent governance?

The ownership, inventory, evaluation, review and incident processes that keep an agent's access and behavior within limits the business has approved, supported by technical controls such as least privilege and logging.

Can prompt injection be fully prevented?

OWASP's 2025 list said it was unclear whether fool-proof prevention exists, its 2026 list says no reliable prevention mechanism exists today, and the NCSC thinks it is unlikely to be mitigated as SQL injection was. The realistic goal is to limit what a manipulated agent can do.

Does the EU AI Act require human oversight of every AI agent?

Not as such: Article 14 and the deployer duties in Article 26 apply to high-risk AI systems, from December 2, 2027 (Annex III) and August 2, 2028 (Annex I). Transparency duties under Article 50 apply from August 2, 2026. Check how your use case is classified.

Do we need ISO/IEC 42001?

Nothing we read makes it mandatory. ISO presents it as a standard that can structure how an organization manages AI.

Where should a small business start?

Inventory the agents, remove permissions they do not need, require approval for money and data changes, turn on logging, and write down how to switch each agent off. If the agent answers calls, what an AI voice agent is explains the moving parts.

In short

An agent's risk depends less on the model than on what it is connected to. Assume the model can be manipulated, give it the narrowest tools and credentials, keep authorization and approvals outside it, log its actions and know how to stop it. OWASP, the NCSC and the May 2026 joint guidance converge on the technical steps, NIST's framework backs the governance ones, and the EU AI Act adds transparency duties from August 2, 2026. To review an agent project, talk to us.

Sources

Review your agent project