AI agents are creating a new dimension of technology risk because, unlike conventional software or standalone large language models, they can be connected to tools that allow them to read information, modify records, send communications or execute actions. A recent analysis by cybersecurity researcher Ibrahim Mukherjee applies the Swiss Cheese Model to AI-agent safety, arguing that organisations should not rely on a single safeguard when deploying agents in critical environments.
The analysis distinguishes between an AI model’s capability and its authority. An agent may understand how to perform a high-value transaction or modify a system without necessarily needing permission to do so. The proposed control architecture includes narrowly scoped tool permissions, dedicated agent identities, human approval for high-risk actions, transaction limits, monitoring, rate controls and independent runtime safeguards. The article also highlights recent approaches from AWS, Microsoft, OpenAI and NVIDIA, including tool-call authorisation, agent identities, approval mechanisms and runtime isolation.
The central risk-management principle is that one control should not be expected to prevent every failure. Multiple independent controls can reduce the likelihood that an unexpected model behaviour becomes a material incident. The analysis also argues that agents should not be able to modify their own permissions, identity controls or safety mechanisms. For banks and other regulated institutions, this reinforces the need to define exactly what an AI agent can access, what it can change, what requires human approval and who remains accountable for its actions.
Want to deepen your expertise beyond today’s news?
Explore practical certification courses designed for banking, risk, insurance, compliance, ESG, AI, and emerging technologies professionals.
Learn from industry experts and earn certifications from RMAI and BFSI Sector Skill Council of India.
#Riskmanagementnews
