Your AI has a Staff Pass. Who The Hell Approved That?

Imagine a new member of staff arriving on Monday morning. Before anyone has explained the expenses policy, they have access to customer records, the finance system and a company credit card. They can also email anybody they like.

Their manager assures you that everything is fine because they performed brilliantly in the interview.

I suspect even the most enthusiastic advocate of employee empowerment would have a few follow-up questions. And probably before lunch.

Yet this is a useful way to look at the permissions we give AI agents. A demonstration shows something finding information, joining it together, and completing a task. Everyone is impressed. Then somebody connects it to the systems where completing that task has actual consequences.

The demonstration has quietly acquired purchasing authority.

An agent is an AI system that can use tools to take steps towards a goal. That might mean looking up an order, changing a booking or initiating a payment. OWASP describes excessive agency as a risk arising from too much functionality, permission or autonomy. The damage depends on what the system can reach and do. Source: OWASP, Excessive Agency.

The useful security question is therefore wonderfully ordinary: what have we authorised this thing to do?

Take a hypothetical customer service agent. Looking up an order is useful. Drafting a response is useful. Issuing refunds might also be useful. Changing bank details because a persuasive paragraph appeared in a support ticket is rather less useful, unless your business model involves generously supporting strangers.

Those activities need different boundaries. They should not inherit the same permissions simply because one tool happens to offer them all.

I would start with the business process. Which actions are necessary? Which records may be accessed? What financial limits apply? Which changes require a second person? What happens when the request is ambiguous?

Then make those answers enforceable in the services the agent uses. A refund limit needs to be checked by the payment workflow. Access to a customer record needs to depend on an appropriate identity and authorisation check. A sentence telling the agent to behave itself can support that design, but the actual transaction still needs a gatekeeper.

Otherwise we have written a very polite policy and attached it to an open door.

Human approval also needs some thought. Presenting somebody with a long conversation and a large green button is an excellent way to generate clicks. It is a poor way to support a decision. Show the proposed action, the affected account, the amount, the destination and anything unusual about the request. Give the reviewer enough context to disagree.

We should also test the awkward journeys. What happens if an instruction arrives inside a document? Can a sequence of small actions exceed a limit that would block one large action? Does the agent retain access after its task has ended? Can it pass work to another agent with broader permissions?

These are questions about the entire workflow. Watching the final answer alone will miss much of what matters. I want evidence of the tool calls, the permissions exercised and the controls that stopped an inappropriate action.

There is a practical incident question here too. If the agent starts behaving unexpectedly, who can stop it? Can they revoke its credentials, suspend a tool or fall back to a manual process? Has anyone tried doing that while the business is busy?

None of this requires us to abandon useful automation. It gives us a basis for allowing more of it. A tightly scoped agent performing a valuable task can be a sensible business decision. An agent with access to everything because the integration was easier that way deserves a different conversation.

Before issuing the staff pass, decide which doors it opens. The impressive interview is already over.

Tags: