When AI Gets Authority
For the last few years, businesses have been concentrating on what generative AI can tell us. AI agents introduce a rather more consequential question: what happens when software can decide what to do next, and then actually do it?
In July 2026, an AI agent being used as part of an OpenAI cybersecurity evaluation compromised part of Hugging Face's production infrastructure. There is some important context here before we get carried away. This was not a normal business application suddenly deciding to attack another company. OpenAI was deliberately testing highly capable models against difficult cybersecurity tasks, and the production classifiers normally used to prevent models pursuing high-risk cyber activity were intentionally not enabled for the evaluation.
Even so, what happened is worth paying attention to. According to the subsequent investigations, the agent appears to have concluded that information held by Hugging Face could help it complete the benchmark it had been given. It found a way out through a zero-day vulnerability in infrastructure used by the evaluation environment, gained wider internet access, chained together other attack paths, obtained credentials and eventually reached Hugging Face systems. Hugging Face's forensic reconstruction covered around 17,600 individual actions, grouped into approximately 6,280 clusters.
The temptation, naturally, is to reach for the "AI escaped" or "AI went rogue" headline. I am not sure either is particularly useful. As far as the investigations could establish, the software was trying to achieve the objective it had been given and found a route nobody expected it to take. Hugging Face believes the agent was effectively trying to cheat the evaluation by reaching production systems where it thought benchmark information or solutions might be available.
That is considerably less theatrical, but from a digital transformation perspective probably more important.
For the last few years, most organisations have experienced generative AI as something they ask. Give it some information, ask a question and get something back. It might summarise a contract, analyse some data, write some code or draft an email, but there is generally still a person sitting on the other side deciding what happens next.
Agents start to blur that boundary. Give the same intelligence access to tools and it can retrieve information, call an API, update a record, execute some code, send the email or make the change itself. OpenAI describes agents in much the same way: systems that can independently manage workflow execution, make decisions, select appropriate tools and interact with external systems on a user's behalf.
The chatbot just got write access.
That shift is why AI agent governance is becoming a practical enterprise concern rather than simply another policy discussion. Once software can take action rather than make recommendations, questions about identity, permissions, controls, auditability and accountability move into the architecture itself.
Automation changes when software is given intent, not instructions
Software has been changing things for decades, so there is a danger of overselling what is new here. We do not need AI to update a database, trigger an API or run a workflow. Scripts, rules engines, orchestration platforms and automated processes have been doing those jobs perfectly well for years.
The difference is in how we get from the objective to the action. Traditional automation is normally designed around a route somebody has described in advance. When X happens, retrieve Y. If the answer is Z, perform this action. There may be hundreds of rules, branches and exceptions involved, but somebody has still tried to model the process beforehand.
Agents potentially allow us to move further towards giving software an outcome rather than every instruction required to reach it. Here is the problem. Work out what information you need. Decide which tools are appropriate. Deal with what comes back and get the job done.
That distinction matters because the execution path can become dynamic. An agent might start with one system, discover it needs some additional context, call something else, encounter a result it was not expecting, revise its approach and carry on. Current agent architectures are explicitly built around this ability to manage a workflow, choose tools and take actions depending on the state of the task.
That is a materially different proposition from conventional automation. The software is no longer simply executing a predetermined flow. Within whatever boundaries we establish around it, we are giving it some discretion over how that flow unfolds.
And that is where this gets much more interesting than another conversation about whether AI can write a better email.
The opportunity is to remove coordination, not people
A surprising amount of work inside an organisation is not difficult because any individual task is particularly complicated. It is difficult because of the coordination required between the tasks.
Open the CRM. Find the customer. Copy an identifier into the billing platform. Read the notes. Check another system. Work out what they mean. Ask another team whether something is normal. Wait for the answer. Update the first platform and send the customer an email.
Sound familiar?
We have spent years trying to automate those processes, and conventional workflow technology is extremely effective when the steps can be described predictably. The difficult bits tend to be the exceptions: the document that is not structured properly, the account that does not quite match, three systems that disagree with one another, or the point in the process where somebody needs to read something and understand what it means before they know what to do next.
Those are precisely the kinds of workflows where agents potentially become useful. OpenAI's current guidance specifically identifies complex decision making, difficult-to-maintain rules and heavy reliance on unstructured information as areas where an agentic approach may offer something that conventional deterministic automation does not. It also makes the important point that where those characteristics are absent, a deterministic solution may still be the better answer.
If software can take over some of the coordination between systems, information and decisions, there is potentially a meaningful efficiency gain. Not because everybody involved in the process suddenly disappears, but because we can stop using people as expensive middleware between platforms that were never particularly good at coordinating with each other.
There is an important distinction there. The objective should not necessarily be to remove people from the organisation. It should be to remove the work we currently make people do simply because the technology underneath them cannot handle context, ambiguity or exceptions very well.
That is why I think the opportunity around agents is much more significant than incremental productivity. Drafting a customer response ten seconds faster is useful. Allowing software to investigate the problem, reconcile the relevant information and carry the process forward is something else entirely.
Of course, the further it carries the process, the closer it eventually gets to an action with consequences.
And that is where things become rather more complicated.
Capability is not authority
A few months before the Hugging Face incident, PocketOS published a candid account of an AI agent deleting the company's production database. The deletion took nine seconds. Recovery took sixty hours. Its backups existed, but were three months out of date because the backup process had quietly stopped working.
PocketOS is also very clear that this was not simply an "AI problem". The company says the agent acted with confidence using permissions it should never have had, against an environment that did not have sufficient friction around the destructive action.
That is the useful lesson.
Humans have been deleting production databases for years, and software bugs have made a respectable contribution too. The interesting part is that the actor had sufficient authority for its mistake to matter.
We already understand this when the actor is human. Someone in customer services might be authorised to issue a £20 refund without anybody caring very much. £200 may still be entirely reasonable. £20,000 probably introduces another conversation.
The same applies elsewhere. An agent identifying a spelling mistake in some content metadata and correcting it is probably fairly benign. If the field it wants to change determines whether that content can be distributed in a particular territory, the consequence is very different. Likewise, an agent may correctly identify a production configuration issue, but that does not mean it should inherit unlimited rights to reconfigure the whole environment.
None of this is particularly exotic. It is identity and access management applied to a new type of actor, and I suspect it will become one of the foundations of effective agentic AI governance.
Microsoft is already treating agents in these terms. Its current guidance recommends a unique, dedicated AI agent identity, a named owner or sponsor, clearly documented purpose and data access, least-privilege access, controlled tool permissions, logging and tested revocation paths. NIST's work in the same area is explicitly examining identification, authorisation, auditing and non-repudiation for AI agents.
In practical terms, that means AI agent permissions need to be treated with the same seriousness as privileged human and service accounts. An agent can be perfectly capable of determining what it thinks should happen without being entitled to make it happen.
That distinction matters. AI agent controls cannot simply ask whether a model is technically capable of performing an action. They need to determine whether this particular agent, acting in this particular context, is authorised to perform it.
As an aside, I suspect AI agent sprawl will become one of those problems organisations do not think they have until somebody finally asks, "How many agents do we actually have?" We have been through variations of this with SaaS applications, cloud resources, service accounts and API keys already. There is no obvious reason the agent estate will organise itself any better, particularly in businesses where experimentation is deliberately devolved across teams.
That does not mean centralising every AI initiative. A central team is unlikely to understand every process well enough to identify all of the useful applications itself. Devolved innovation can be entirely sensible.
Devolved authority by accident is a different matter.
Agentic AI inherits every compromise in your digital estate
There is another awkward issue underneath all of this. Your agent is going to inherit your existing digital estate.
The shiny APIs, the legacy systems, the inconsistent customer records, the integration somebody wrote eight years ago and nobody wants to touch, and the platform that calls a customer one thing while another platform calls them something else. A company grows, platforms get bought, teams solve individual problems, acquisitions happen, suppliers change and priorities move. Eventually you end up with what everyone politely refers to as the current architecture.
Now imagine an agent dealing with a customer who has paid for a service but cannot access it. The answer might require information from CRM, billing, authentication, entitlement and support systems before anyone can determine what has actually happened.
A human operator will often deal with the inconsistencies almost without thinking about them. They know one system updates before another. They notice that the identifiers are slightly different. Perhaps they remember that customers on an old package behave differently, or simply ask the person sitting next to them because the documentation has not been updated since 2023.
Give the job to an agent and those assumptions need to exist somewhere. Which system is authoritative? What happens if billing says the payment succeeded but the entitlement platform says the account is inactive? Is the information wrong or is there a business rule the agent does not understand? Is one platform simply five minutes behind?
These are old data and architecture problems with a new consumer. The material difference is that the consumer may now take action on whatever conclusion it reaches.
Poor data feeding a chatbot can produce a bad answer. Poor data feeding autonomous software can produce a bad outcome.
There is another complication too. The information does not necessarily have to be wrong by accident. Research published in July demonstrated agent data injection attacks in which attacker-controlled information could be presented as apparently legitimate contextual data and influence what an agent subsequently did. The researchers identified vulnerabilities in real web and coding agents that could result in unintended clicks, remote code execution and software supply-chain actions.
That creates an uncomfortable crossover between data architecture and security. If information can influence the action, where that information came from, whether it can be trusted and how much weight the agent should give it are no longer abstract governance concerns. They become part of the execution model.
This is also why I would be careful with the idea that agents somehow make technical debt less important. An agent may make fragmented environments easier to navigate because it can reason about a task rather than rely entirely on a fixed integration path, but it still needs access to systems, usable interfaces, meaningful data, appropriate permissions and some idea of what to do when something fails halfway through.
If the API does not exist, the agent needs another way in. If the same customer has six identifiers, somebody or something still needs to establish whether they represent the same person. If an undocumented business rule matters to the decision, that knowledge needs to be available somewhere.
Putting an intelligent orchestration layer in front of a messy estate does not make the estate stop being messy. It might make the mess easier to navigate. It may also allow the consequences of that mess to propagate considerably faster.
Human oversight has an economic cost
At this point in most conversations somebody will say there will always be a human in the loop.
That sounds reassuring, but it raises an obvious question: if somebody still needs to review and approve everything the agent does, how much have we actually automated?
Imagine our customer service agent has checked five systems, identified an account problem, established that the customer was incorrectly charged and calculated that a £20 refund is appropriate. It then places the whole thing into a queue for somebody to click "Approve".
There may still be a meaningful saving because the investigation has been automated, but run that process across tens of thousands of transactions and you have retained a sizeable manual operation. The bottleneck has moved rather than disappeared.
This is where some of the safer language around agentic AI starts to conflict with the economics of automation. We introduce autonomy because we want software to remove work from a process, then become sufficiently nervous about the autonomy that we put a person back into every transaction.
Eventually that rather defeats the point.
The challenge with AI agent oversight is therefore that more oversight is not automatically better. Every approval step may reduce a particular type of risk, but it also reintroduces cost, latency and manual effort into the process. The better question is where human judgement genuinely adds value.
If the refund is £20, the rules are clear, all of the relevant systems agree and the action can easily be reversed, perhaps nobody needs to approve it. If the amount is £20,000, two systems disagree or the situation falls outside normal parameters, that sounds like an excellent time to involve someone.
For many processes, the useful destination is therefore probably not human in the loop but human by exception. Routine, well-understood work becomes increasingly autonomous, while unusual, ambiguous or high-consequence cases are escalated.
That still leaves plenty of room to introduce autonomy gradually. An agent can begin by observing and recommending. Once its behaviour is understood, it can prepare actions for approval. Eventually, specific classes of routine and reversible action can be executed automatically, with people becoming involved when the process crosses an agreed threshold.
OpenAI's guidance takes a similar risk-based approach. It recommends human intervention when agents exceed defined failure thresholds and around actions that are sensitive, irreversible or high stakes, such as large refunds or payments.
PocketOS arrived at the same issue from a considerably less theoretical direction. Following its database incident, destructive operations now require explicit human confirmation. The company did not respond by abandoning autonomous agents. It says it continues to run a number of narrow autonomous agents in production, while changing where the control boundary sits.
That seems to me the more useful way to think about human oversight. It is a control that should be applied where the consequence warrants it, not necessarily a permanent operating model for every action performed by autonomous software.
The point of automation is, after all, to automate. The design problem is deciding where people genuinely improve the outcome and where they are simply being retained because we do not yet trust the system enough to remove them.
Autonomy without traceability is operationally indefensible
If we move towards humans by exception, something else becomes more important rather than less: understanding what the agent actually did.
The Hugging Face investigation is a useful extreme example. Its forensic reconstruction covered roughly 17,600 actions across thousands of clusters, with investigators piecing together how the agent moved through systems and adapted its behaviour over time.
Bring that down to a normal business process. A customer says something changed incorrectly on their account and you discover an agent made the change.
"The AI did it" is unlikely to get you very far in the incident review.
You will want to know which agent acted, who initiated the task, which information it used, what that information contained at the time, which tools it called and which permission ultimately allowed the change. If another agent became involved along the way, that probably matters too.
This is where AI agent auditability stops being a governance nicety and becomes an operational requirement. The less direct human involvement there is in execution, the stronger the surrounding observability needs to become.
Microsoft's current guidance recommends logging the agent identity, its role and effective scope, the action performed, the affected resource and any user on whose behalf it was acting. NIST is likewise explicitly considering auditing and non-repudiation as part of the identity and authorisation problem for agents.
That does not mean every autonomous action needs somebody watching it. We already operate enormous automated environments without putting a person in front of every transaction, infrastructure event or network request. We establish limits, controls and monitoring, then investigate the things that fall outside them.
There is no obvious reason mature agentic systems should work differently.
Autonomy does not remove oversight. It changes when oversight happens.
Delegated intent complicates the trust boundary
There is one further development worth watching because it makes this problem more interesting again.
Agents are increasingly being designed to communicate and collaborate with other agents. Google's Agent2Agent protocol standardises how agents can discover and communicate with one another, including agents built by different teams and on different technology stacks. Its newer Agentic Resource Discovery work addresses the related question of how agents can find and verify tools, skills and other agents distributed across teams, organisations and platforms.
Imagine an internal agent receives a task and determines that another specialist agent can perform part of it. That second agent uses a tool which calls one of your services and ultimately changes something.
Who is acting?
The second agent? The first agent? The person who initiated the original task?
If the first agent is authorised to perform an action, can it delegate that authority? Does the second agent receive all of it, part of it or none of it? What happens if the agents belong to different organisations?
These are emerging questions and it would be premature to pretend the industry has settled on all of the answers. NIST's AI Agent Standards Initiative is explicitly conducting work around agent authentication and identity infrastructure to support secure human-agent and multi-agent interactions.
But the direction matters. Historically, trust has often been associated with a known user, application or integration. Once agents can dynamically discover capabilities and delegate parts of an objective elsewhere, trust and authority need to survive the journey.
The task can be dynamic. Accountability cannot disappear every time it crosses another boundary.
Agentic transformation should start with delegation, not technology
There is a pattern with significant technology shifts that most people involved in transformation will recognise. Something becomes strategically important, somebody decides the organisation needs it and everyone starts searching for somewhere to put it.
We need an AI strategy. We need some agent use cases. We should run a workshop. Which processes can we make agentic?
I would probably approach it the other way around.
Start with the process that annoys everybody. The one that is slow, expensive, fragmented or depends on somebody transferring information between systems all day. Understand where the decisions are made, what information those decisions rely on, which systems are involved, where the awkward exceptions occur and what happens when something goes wrong.
Then look at what the people in the process are actually doing. Are they exercising valuable judgement, or are they compensating for limitations in the technology around them? Are they approving something because there is genuine financial or operational risk, or because the workflow cannot distinguish a routine case from an unusual one? Are they manually reconciling information because the organisation genuinely needs their expertise, or because two systems use different identifiers?
Those are quite different problems.
Once you understand that, decide what you are prepared to delegate. Which decisions can software make? Which actions can it take? Which are reversible? Where do you need escalation? What authority does it require? What evidence would you need afterwards if somebody asked why a decision was made?
Only then does the technology choice become particularly interesting.
An agent may be an excellent answer. It may not. That is fine. Current guidance on agent design makes much the same distinction: agents are well suited to workflows where ambiguity, complex decisions or unstructured data cause deterministic approaches to struggle, but where those characteristics are absent, conventional automation may remain entirely sufficient.
The objective is not to deploy agents. It is to improve the business.
The architecture of automation is becoming the architecture of authority
For the last twenty years, digital transformation has largely involved connecting people to systems and systems to one another. We are now starting to introduce another participant into that estate: software that can receive an objective, determine how to pursue it and use those systems on our behalf.
There are some genuinely interesting possibilities in that, particularly around the coordination work that has resisted conventional automation. NIST itself describes the potential for agents to improve productivity, efficiency and decision making, while highlighting the need for appropriate identification and authorisation controls when those agents are given access to organisational data, tools and applications.
I think that combination is important. The opportunity and the risk come from the same characteristic: the agent has some freedom to decide how work gets done.
Once that happens, architecture is no longer concerned only with whether one system can connect to another. It increasingly determines which autonomous actor may use which information, exercise which capability, under whose authority, with what limits and with what evidence left behind.
In practice, enterprise AI agent governance is therefore unlikely to be solved by another policy document sitting somewhere on the intranet. It has to show up in the identities we create, the permissions we grant, the controls around consequential actions, the quality and provenance of the data agents consume, the escalation paths we design and the audit evidence we retain afterwards.
Hugging Face gives us an extreme example of autonomous software finding a route its operators did not expect. PocketOS gives us a much more everyday example of what happens when an agent simply has more authority than the task requires. The emerging work around agent identity, least privilege, secure multi-agent interaction and trusted resource discovery shows that the surrounding control model is already having to catch up with the capability.
None of that is an argument against agentic AI. Quite the opposite. If agents can remove some of the human coordination that makes modern digital businesses expensive and cumbersome to operate, there is a very tangible reason to pursue them.
But introducing autonomous actors into an existing digital estate is not simply another AI implementation. It changes assumptions about identity, trust, data, accountability and, ultimately, authority.
So before asking what an agent can do, I would start with a different question.
What are you actually prepared to let it do?
When AI Gets Authority
For the last few years, businesses have been concentrating on what generative AI can tell us. AI agents introduce a rather more consequential question: what happens when software can decide what to do next, and then actually do it?
In July 2026, an AI agent being used as part of an OpenAI cybersecurity evaluation compromised part of Hugging Face's production infrastructure. There is some important context here before we get carried away. This was not a normal business application suddenly deciding to attack another company. OpenAI was deliberately testing highly capable models against difficult cybersecurity tasks, and the production classifiers normally used to prevent models pursuing high-risk cyber activity were intentionally not enabled for the evaluation.
Even so, what happened is worth paying attention to. According to the subsequent investigations, the agent appears to have concluded that information held by Hugging Face could help it complete the benchmark it had been given. It found a way out through a zero-day vulnerability in infrastructure used by the evaluation environment, gained wider internet access, chained together other attack paths, obtained credentials and eventually reached Hugging Face systems. Hugging Face's forensic reconstruction covered around 17,600 individual actions, grouped into approximately 6,280 clusters.
The temptation, naturally, is to reach for the "AI escaped" or "AI went rogue" headline. I am not sure either is particularly useful. As far as the investigations could establish, the software was trying to achieve the objective it had been given and found a route nobody expected it to take. Hugging Face believes the agent was effectively trying to cheat the evaluation by reaching production systems where it thought benchmark information or solutions might be available.
That is considerably less theatrical, but from a digital transformation perspective probably more important.
For the last few years, most organisations have experienced generative AI as something they ask. Give it some information, ask a question and get something back. It might summarise a contract, analyse some data, write some code or draft an email, but there is generally still a person sitting on the other side deciding what happens next.
Agents start to blur that boundary. Give the same intelligence access to tools and it can retrieve information, call an API, update a record, execute some code, send the email or make the change itself. OpenAI describes agents in much the same way: systems that can independently manage workflow execution, make decisions, select appropriate tools and interact with external systems on a user's behalf.
The chatbot just got write access.
That shift is why AI agent governance is becoming a practical enterprise concern rather than simply another policy discussion. Once software can take action rather than make recommendations, questions about identity, permissions, controls, auditability and accountability move into the architecture itself.
Automation changes when software is given intent, not instructions
Software has been changing things for decades, so there is a danger of overselling what is new here. We do not need AI to update a database, trigger an API or run a workflow. Scripts, rules engines, orchestration platforms and automated processes have been doing those jobs perfectly well for years.
The difference is in how we get from the objective to the action. Traditional automation is normally designed around a route somebody has described in advance. When X happens, retrieve Y. If the answer is Z, perform this action. There may be hundreds of rules, branches and exceptions involved, but somebody has still tried to model the process beforehand.
Agents potentially allow us to move further towards giving software an outcome rather than every instruction required to reach it. Here is the problem. Work out what information you need. Decide which tools are appropriate. Deal with what comes back and get the job done.
That distinction matters because the execution path can become dynamic. An agent might start with one system, discover it needs some additional context, call something else, encounter a result it was not expecting, revise its approach and carry on. Current agent architectures are explicitly built around this ability to manage a workflow, choose tools and take actions depending on the state of the task.
That is a materially different proposition from conventional automation. The software is no longer simply executing a predetermined flow. Within whatever boundaries we establish around it, we are giving it some discretion over how that flow unfolds.
And that is where this gets much more interesting than another conversation about whether AI can write a better email.
The opportunity is to remove coordination, not people
A surprising amount of work inside an organisation is not difficult because any individual task is particularly complicated. It is difficult because of the coordination required between the tasks.
Open the CRM. Find the customer. Copy an identifier into the billing platform. Read the notes. Check another system. Work out what they mean. Ask another team whether something is normal. Wait for the answer. Update the first platform and send the customer an email.
Sound familiar?
We have spent years trying to automate those processes, and conventional workflow technology is extremely effective when the steps can be described predictably. The difficult bits tend to be the exceptions: the document that is not structured properly, the account that does not quite match, three systems that disagree with one another, or the point in the process where somebody needs to read something and understand what it means before they know what to do next.
Those are precisely the kinds of workflows where agents potentially become useful. OpenAI's current guidance specifically identifies complex decision making, difficult-to-maintain rules and heavy reliance on unstructured information as areas where an agentic approach may offer something that conventional deterministic automation does not. It also makes the important point that where those characteristics are absent, a deterministic solution may still be the better answer.
If software can take over some of the coordination between systems, information and decisions, there is potentially a meaningful efficiency gain. Not because everybody involved in the process suddenly disappears, but because we can stop using people as expensive middleware between platforms that were never particularly good at coordinating with each other.
There is an important distinction there. The objective should not necessarily be to remove people from the organisation. It should be to remove the work we currently make people do simply because the technology underneath them cannot handle context, ambiguity or exceptions very well.
That is why I think the opportunity around agents is much more significant than incremental productivity. Drafting a customer response ten seconds faster is useful. Allowing software to investigate the problem, reconcile the relevant information and carry the process forward is something else entirely.
Of course, the further it carries the process, the closer it eventually gets to an action with consequences.
And that is where things become rather more complicated.
Capability is not authority
A few months before the Hugging Face incident, PocketOS published a candid account of an AI agent deleting the company's production database. The deletion took nine seconds. Recovery took sixty hours. Its backups existed, but were three months out of date because the backup process had quietly stopped working.
PocketOS is also very clear that this was not simply an "AI problem". The company says the agent acted with confidence using permissions it should never have had, against an environment that did not have sufficient friction around the destructive action.
That is the useful lesson.
Humans have been deleting production databases for years, and software bugs have made a respectable contribution too. The interesting part is that the actor had sufficient authority for its mistake to matter.
We already understand this when the actor is human. Someone in customer services might be authorised to issue a £20 refund without anybody caring very much. £200 may still be entirely reasonable. £20,000 probably introduces another conversation.
The same applies elsewhere. An agent identifying a spelling mistake in some content metadata and correcting it is probably fairly benign. If the field it wants to change determines whether that content can be distributed in a particular territory, the consequence is very different. Likewise, an agent may correctly identify a production configuration issue, but that does not mean it should inherit unlimited rights to reconfigure the whole environment.
None of this is particularly exotic. It is identity and access management applied to a new type of actor, and I suspect it will become one of the foundations of effective agentic AI governance.
Microsoft is already treating agents in these terms. Its current guidance recommends a unique, dedicated AI agent identity, a named owner or sponsor, clearly documented purpose and data access, least-privilege access, controlled tool permissions, logging and tested revocation paths. NIST's work in the same area is explicitly examining identification, authorisation, auditing and non-repudiation for AI agents.
In practical terms, that means AI agent permissions need to be treated with the same seriousness as privileged human and service accounts. An agent can be perfectly capable of determining what it thinks should happen without being entitled to make it happen.
That distinction matters. AI agent controls cannot simply ask whether a model is technically capable of performing an action. They need to determine whether this particular agent, acting in this particular context, is authorised to perform it.
As an aside, I suspect AI agent sprawl will become one of those problems organisations do not think they have until somebody finally asks, "How many agents do we actually have?" We have been through variations of this with SaaS applications, cloud resources, service accounts and API keys already. There is no obvious reason the agent estate will organise itself any better, particularly in businesses where experimentation is deliberately devolved across teams.
That does not mean centralising every AI initiative. A central team is unlikely to understand every process well enough to identify all of the useful applications itself. Devolved innovation can be entirely sensible.
Devolved authority by accident is a different matter.
Agentic AI inherits every compromise in your digital estate
There is another awkward issue underneath all of this. Your agent is going to inherit your existing digital estate.
The shiny APIs, the legacy systems, the inconsistent customer records, the integration somebody wrote eight years ago and nobody wants to touch, and the platform that calls a customer one thing while another platform calls them something else. A company grows, platforms get bought, teams solve individual problems, acquisitions happen, suppliers change and priorities move. Eventually you end up with what everyone politely refers to as the current architecture.
Now imagine an agent dealing with a customer who has paid for a service but cannot access it. The answer might require information from CRM, billing, authentication, entitlement and support systems before anyone can determine what has actually happened.
A human operator will often deal with the inconsistencies almost without thinking about them. They know one system updates before another. They notice that the identifiers are slightly different. Perhaps they remember that customers on an old package behave differently, or simply ask the person sitting next to them because the documentation has not been updated since 2023.
Give the job to an agent and those assumptions need to exist somewhere. Which system is authoritative? What happens if billing says the payment succeeded but the entitlement platform says the account is inactive? Is the information wrong or is there a business rule the agent does not understand? Is one platform simply five minutes behind?
These are old data and architecture problems with a new consumer. The material difference is that the consumer may now take action on whatever conclusion it reaches.
Poor data feeding a chatbot can produce a bad answer. Poor data feeding autonomous software can produce a bad outcome.
There is another complication too. The information does not necessarily have to be wrong by accident. Research published in July demonstrated agent data injection attacks in which attacker-controlled information could be presented as apparently legitimate contextual data and influence what an agent subsequently did. The researchers identified vulnerabilities in real web and coding agents that could result in unintended clicks, remote code execution and software supply-chain actions.
That creates an uncomfortable crossover between data architecture and security. If information can influence the action, where that information came from, whether it can be trusted and how much weight the agent should give it are no longer abstract governance concerns. They become part of the execution model.
This is also why I would be careful with the idea that agents somehow make technical debt less important. An agent may make fragmented environments easier to navigate because it can reason about a task rather than rely entirely on a fixed integration path, but it still needs access to systems, usable interfaces, meaningful data, appropriate permissions and some idea of what to do when something fails halfway through.
If the API does not exist, the agent needs another way in. If the same customer has six identifiers, somebody or something still needs to establish whether they represent the same person. If an undocumented business rule matters to the decision, that knowledge needs to be available somewhere.
Putting an intelligent orchestration layer in front of a messy estate does not make the estate stop being messy. It might make the mess easier to navigate. It may also allow the consequences of that mess to propagate considerably faster.
Human oversight has an economic cost
At this point in most conversations somebody will say there will always be a human in the loop.
That sounds reassuring, but it raises an obvious question: if somebody still needs to review and approve everything the agent does, how much have we actually automated?
Imagine our customer service agent has checked five systems, identified an account problem, established that the customer was incorrectly charged and calculated that a £20 refund is appropriate. It then places the whole thing into a queue for somebody to click "Approve".
There may still be a meaningful saving because the investigation has been automated, but run that process across tens of thousands of transactions and you have retained a sizeable manual operation. The bottleneck has moved rather than disappeared.
This is where some of the safer language around agentic AI starts to conflict with the economics of automation. We introduce autonomy because we want software to remove work from a process, then become sufficiently nervous about the autonomy that we put a person back into every transaction.
Eventually that rather defeats the point.
The challenge with AI agent oversight is therefore that more oversight is not automatically better. Every approval step may reduce a particular type of risk, but it also reintroduces cost, latency and manual effort into the process. The better question is where human judgement genuinely adds value.
If the refund is £20, the rules are clear, all of the relevant systems agree and the action can easily be reversed, perhaps nobody needs to approve it. If the amount is £20,000, two systems disagree or the situation falls outside normal parameters, that sounds like an excellent time to involve someone.
For many processes, the useful destination is therefore probably not human in the loop but human by exception. Routine, well-understood work becomes increasingly autonomous, while unusual, ambiguous or high-consequence cases are escalated.
That still leaves plenty of room to introduce autonomy gradually. An agent can begin by observing and recommending. Once its behaviour is understood, it can prepare actions for approval. Eventually, specific classes of routine and reversible action can be executed automatically, with people becoming involved when the process crosses an agreed threshold.
OpenAI's guidance takes a similar risk-based approach. It recommends human intervention when agents exceed defined failure thresholds and around actions that are sensitive, irreversible or high stakes, such as large refunds or payments.
PocketOS arrived at the same issue from a considerably less theoretical direction. Following its database incident, destructive operations now require explicit human confirmation. The company did not respond by abandoning autonomous agents. It says it continues to run a number of narrow autonomous agents in production, while changing where the control boundary sits.
That seems to me the more useful way to think about human oversight. It is a control that should be applied where the consequence warrants it, not necessarily a permanent operating model for every action performed by autonomous software.
The point of automation is, after all, to automate. The design problem is deciding where people genuinely improve the outcome and where they are simply being retained because we do not yet trust the system enough to remove them.
Autonomy without traceability is operationally indefensible
If we move towards humans by exception, something else becomes more important rather than less: understanding what the agent actually did.
The Hugging Face investigation is a useful extreme example. Its forensic reconstruction covered roughly 17,600 actions across thousands of clusters, with investigators piecing together how the agent moved through systems and adapted its behaviour over time.
Bring that down to a normal business process. A customer says something changed incorrectly on their account and you discover an agent made the change.
"The AI did it" is unlikely to get you very far in the incident review.
You will want to know which agent acted, who initiated the task, which information it used, what that information contained at the time, which tools it called and which permission ultimately allowed the change. If another agent became involved along the way, that probably matters too.
This is where AI agent auditability stops being a governance nicety and becomes an operational requirement. The less direct human involvement there is in execution, the stronger the surrounding observability needs to become.
Microsoft's current guidance recommends logging the agent identity, its role and effective scope, the action performed, the affected resource and any user on whose behalf it was acting. NIST is likewise explicitly considering auditing and non-repudiation as part of the identity and authorisation problem for agents.
That does not mean every autonomous action needs somebody watching it. We already operate enormous automated environments without putting a person in front of every transaction, infrastructure event or network request. We establish limits, controls and monitoring, then investigate the things that fall outside them.
There is no obvious reason mature agentic systems should work differently.
Autonomy does not remove oversight. It changes when oversight happens.
Delegated intent complicates the trust boundary
There is one further development worth watching because it makes this problem more interesting again.
Agents are increasingly being designed to communicate and collaborate with other agents. Google's Agent2Agent protocol standardises how agents can discover and communicate with one another, including agents built by different teams and on different technology stacks. Its newer Agentic Resource Discovery work addresses the related question of how agents can find and verify tools, skills and other agents distributed across teams, organisations and platforms.
Imagine an internal agent receives a task and determines that another specialist agent can perform part of it. That second agent uses a tool which calls one of your services and ultimately changes something.
Who is acting?
The second agent? The first agent? The person who initiated the original task?
If the first agent is authorised to perform an action, can it delegate that authority? Does the second agent receive all of it, part of it or none of it? What happens if the agents belong to different organisations?
These are emerging questions and it would be premature to pretend the industry has settled on all of the answers. NIST's AI Agent Standards Initiative is explicitly conducting work around agent authentication and identity infrastructure to support secure human-agent and multi-agent interactions.
But the direction matters. Historically, trust has often been associated with a known user, application or integration. Once agents can dynamically discover capabilities and delegate parts of an objective elsewhere, trust and authority need to survive the journey.
The task can be dynamic. Accountability cannot disappear every time it crosses another boundary.
Agentic transformation should start with delegation, not technology
There is a pattern with significant technology shifts that most people involved in transformation will recognise. Something becomes strategically important, somebody decides the organisation needs it and everyone starts searching for somewhere to put it.
We need an AI strategy. We need some agent use cases. We should run a workshop. Which processes can we make agentic?
I would probably approach it the other way around.
Start with the process that annoys everybody. The one that is slow, expensive, fragmented or depends on somebody transferring information between systems all day. Understand where the decisions are made, what information those decisions rely on, which systems are involved, where the awkward exceptions occur and what happens when something goes wrong.
Then look at what the people in the process are actually doing. Are they exercising valuable judgement, or are they compensating for limitations in the technology around them? Are they approving something because there is genuine financial or operational risk, or because the workflow cannot distinguish a routine case from an unusual one? Are they manually reconciling information because the organisation genuinely needs their expertise, or because two systems use different identifiers?
Those are quite different problems.
Once you understand that, decide what you are prepared to delegate. Which decisions can software make? Which actions can it take? Which are reversible? Where do you need escalation? What authority does it require? What evidence would you need afterwards if somebody asked why a decision was made?
Only then does the technology choice become particularly interesting.
An agent may be an excellent answer. It may not. That is fine. Current guidance on agent design makes much the same distinction: agents are well suited to workflows where ambiguity, complex decisions or unstructured data cause deterministic approaches to struggle, but where those characteristics are absent, conventional automation may remain entirely sufficient.
The objective is not to deploy agents. It is to improve the business.
The architecture of automation is becoming the architecture of authority
For the last twenty years, digital transformation has largely involved connecting people to systems and systems to one another. We are now starting to introduce another participant into that estate: software that can receive an objective, determine how to pursue it and use those systems on our behalf.
There are some genuinely interesting possibilities in that, particularly around the coordination work that has resisted conventional automation. NIST itself describes the potential for agents to improve productivity, efficiency and decision making, while highlighting the need for appropriate identification and authorisation controls when those agents are given access to organisational data, tools and applications.
I think that combination is important. The opportunity and the risk come from the same characteristic: the agent has some freedom to decide how work gets done.
Once that happens, architecture is no longer concerned only with whether one system can connect to another. It increasingly determines which autonomous actor may use which information, exercise which capability, under whose authority, with what limits and with what evidence left behind.
In practice, enterprise AI agent governance is therefore unlikely to be solved by another policy document sitting somewhere on the intranet. It has to show up in the identities we create, the permissions we grant, the controls around consequential actions, the quality and provenance of the data agents consume, the escalation paths we design and the audit evidence we retain afterwards.
Hugging Face gives us an extreme example of autonomous software finding a route its operators did not expect. PocketOS gives us a much more everyday example of what happens when an agent simply has more authority than the task requires. The emerging work around agent identity, least privilege, secure multi-agent interaction and trusted resource discovery shows that the surrounding control model is already having to catch up with the capability.
None of that is an argument against agentic AI. Quite the opposite. If agents can remove some of the human coordination that makes modern digital businesses expensive and cumbersome to operate, there is a very tangible reason to pursue them.
But introducing autonomous actors into an existing digital estate is not simply another AI implementation. It changes assumptions about identity, trust, data, accountability and, ultimately, authority.
So before asking what an agent can do, I would start with a different question.
What are you actually prepared to let it do?
When AI Gets Authority
For the last few years, businesses have been concentrating on what generative AI can tell us. AI agents introduce a rather more consequential question: what happens when software can decide what to do next, and then actually do it?
In July 2026, an AI agent being used as part of an OpenAI cybersecurity evaluation compromised part of Hugging Face's production infrastructure. There is some important context here before we get carried away. This was not a normal business application suddenly deciding to attack another company. OpenAI was deliberately testing highly capable models against difficult cybersecurity tasks, and the production classifiers normally used to prevent models pursuing high-risk cyber activity were intentionally not enabled for the evaluation.
Even so, what happened is worth paying attention to. According to the subsequent investigations, the agent appears to have concluded that information held by Hugging Face could help it complete the benchmark it had been given. It found a way out through a zero-day vulnerability in infrastructure used by the evaluation environment, gained wider internet access, chained together other attack paths, obtained credentials and eventually reached Hugging Face systems. Hugging Face's forensic reconstruction covered around 17,600 individual actions, grouped into approximately 6,280 clusters.
The temptation, naturally, is to reach for the "AI escaped" or "AI went rogue" headline. I am not sure either is particularly useful. As far as the investigations could establish, the software was trying to achieve the objective it had been given and found a route nobody expected it to take. Hugging Face believes the agent was effectively trying to cheat the evaluation by reaching production systems where it thought benchmark information or solutions might be available.
That is considerably less theatrical, but from a digital transformation perspective probably more important.
For the last few years, most organisations have experienced generative AI as something they ask. Give it some information, ask a question and get something back. It might summarise a contract, analyse some data, write some code or draft an email, but there is generally still a person sitting on the other side deciding what happens next.
Agents start to blur that boundary. Give the same intelligence access to tools and it can retrieve information, call an API, update a record, execute some code, send the email or make the change itself. OpenAI describes agents in much the same way: systems that can independently manage workflow execution, make decisions, select appropriate tools and interact with external systems on a user's behalf.
The chatbot just got write access.
That shift is why AI agent governance is becoming a practical enterprise concern rather than simply another policy discussion. Once software can take action rather than make recommendations, questions about identity, permissions, controls, auditability and accountability move into the architecture itself.
Automation changes when software is given intent, not instructions
Software has been changing things for decades, so there is a danger of overselling what is new here. We do not need AI to update a database, trigger an API or run a workflow. Scripts, rules engines, orchestration platforms and automated processes have been doing those jobs perfectly well for years.
The difference is in how we get from the objective to the action. Traditional automation is normally designed around a route somebody has described in advance. When X happens, retrieve Y. If the answer is Z, perform this action. There may be hundreds of rules, branches and exceptions involved, but somebody has still tried to model the process beforehand.
Agents potentially allow us to move further towards giving software an outcome rather than every instruction required to reach it. Here is the problem. Work out what information you need. Decide which tools are appropriate. Deal with what comes back and get the job done.
That distinction matters because the execution path can become dynamic. An agent might start with one system, discover it needs some additional context, call something else, encounter a result it was not expecting, revise its approach and carry on. Current agent architectures are explicitly built around this ability to manage a workflow, choose tools and take actions depending on the state of the task.
That is a materially different proposition from conventional automation. The software is no longer simply executing a predetermined flow. Within whatever boundaries we establish around it, we are giving it some discretion over how that flow unfolds.
And that is where this gets much more interesting than another conversation about whether AI can write a better email.
The opportunity is to remove coordination, not people
A surprising amount of work inside an organisation is not difficult because any individual task is particularly complicated. It is difficult because of the coordination required between the tasks.
Open the CRM. Find the customer. Copy an identifier into the billing platform. Read the notes. Check another system. Work out what they mean. Ask another team whether something is normal. Wait for the answer. Update the first platform and send the customer an email.
Sound familiar?
We have spent years trying to automate those processes, and conventional workflow technology is extremely effective when the steps can be described predictably. The difficult bits tend to be the exceptions: the document that is not structured properly, the account that does not quite match, three systems that disagree with one another, or the point in the process where somebody needs to read something and understand what it means before they know what to do next.
Those are precisely the kinds of workflows where agents potentially become useful. OpenAI's current guidance specifically identifies complex decision making, difficult-to-maintain rules and heavy reliance on unstructured information as areas where an agentic approach may offer something that conventional deterministic automation does not. It also makes the important point that where those characteristics are absent, a deterministic solution may still be the better answer.
If software can take over some of the coordination between systems, information and decisions, there is potentially a meaningful efficiency gain. Not because everybody involved in the process suddenly disappears, but because we can stop using people as expensive middleware between platforms that were never particularly good at coordinating with each other.
There is an important distinction there. The objective should not necessarily be to remove people from the organisation. It should be to remove the work we currently make people do simply because the technology underneath them cannot handle context, ambiguity or exceptions very well.
That is why I think the opportunity around agents is much more significant than incremental productivity. Drafting a customer response ten seconds faster is useful. Allowing software to investigate the problem, reconcile the relevant information and carry the process forward is something else entirely.
Of course, the further it carries the process, the closer it eventually gets to an action with consequences.
And that is where things become rather more complicated.
Capability is not authority
A few months before the Hugging Face incident, PocketOS published a candid account of an AI agent deleting the company's production database. The deletion took nine seconds. Recovery took sixty hours. Its backups existed, but were three months out of date because the backup process had quietly stopped working.
PocketOS is also very clear that this was not simply an "AI problem". The company says the agent acted with confidence using permissions it should never have had, against an environment that did not have sufficient friction around the destructive action.
That is the useful lesson.
Humans have been deleting production databases for years, and software bugs have made a respectable contribution too. The interesting part is that the actor had sufficient authority for its mistake to matter.
We already understand this when the actor is human. Someone in customer services might be authorised to issue a £20 refund without anybody caring very much. £200 may still be entirely reasonable. £20,000 probably introduces another conversation.
The same applies elsewhere. An agent identifying a spelling mistake in some content metadata and correcting it is probably fairly benign. If the field it wants to change determines whether that content can be distributed in a particular territory, the consequence is very different. Likewise, an agent may correctly identify a production configuration issue, but that does not mean it should inherit unlimited rights to reconfigure the whole environment.
None of this is particularly exotic. It is identity and access management applied to a new type of actor, and I suspect it will become one of the foundations of effective agentic AI governance.
Microsoft is already treating agents in these terms. Its current guidance recommends a unique, dedicated AI agent identity, a named owner or sponsor, clearly documented purpose and data access, least-privilege access, controlled tool permissions, logging and tested revocation paths. NIST's work in the same area is explicitly examining identification, authorisation, auditing and non-repudiation for AI agents.
In practical terms, that means AI agent permissions need to be treated with the same seriousness as privileged human and service accounts. An agent can be perfectly capable of determining what it thinks should happen without being entitled to make it happen.
That distinction matters. AI agent controls cannot simply ask whether a model is technically capable of performing an action. They need to determine whether this particular agent, acting in this particular context, is authorised to perform it.
As an aside, I suspect AI agent sprawl will become one of those problems organisations do not think they have until somebody finally asks, "How many agents do we actually have?" We have been through variations of this with SaaS applications, cloud resources, service accounts and API keys already. There is no obvious reason the agent estate will organise itself any better, particularly in businesses where experimentation is deliberately devolved across teams.
That does not mean centralising every AI initiative. A central team is unlikely to understand every process well enough to identify all of the useful applications itself. Devolved innovation can be entirely sensible.
Devolved authority by accident is a different matter.
Agentic AI inherits every compromise in your digital estate
There is another awkward issue underneath all of this. Your agent is going to inherit your existing digital estate.
The shiny APIs, the legacy systems, the inconsistent customer records, the integration somebody wrote eight years ago and nobody wants to touch, and the platform that calls a customer one thing while another platform calls them something else. A company grows, platforms get bought, teams solve individual problems, acquisitions happen, suppliers change and priorities move. Eventually you end up with what everyone politely refers to as the current architecture.
Now imagine an agent dealing with a customer who has paid for a service but cannot access it. The answer might require information from CRM, billing, authentication, entitlement and support systems before anyone can determine what has actually happened.
A human operator will often deal with the inconsistencies almost without thinking about them. They know one system updates before another. They notice that the identifiers are slightly different. Perhaps they remember that customers on an old package behave differently, or simply ask the person sitting next to them because the documentation has not been updated since 2023.
Give the job to an agent and those assumptions need to exist somewhere. Which system is authoritative? What happens if billing says the payment succeeded but the entitlement platform says the account is inactive? Is the information wrong or is there a business rule the agent does not understand? Is one platform simply five minutes behind?
These are old data and architecture problems with a new consumer. The material difference is that the consumer may now take action on whatever conclusion it reaches.
Poor data feeding a chatbot can produce a bad answer. Poor data feeding autonomous software can produce a bad outcome.
There is another complication too. The information does not necessarily have to be wrong by accident. Research published in July demonstrated agent data injection attacks in which attacker-controlled information could be presented as apparently legitimate contextual data and influence what an agent subsequently did. The researchers identified vulnerabilities in real web and coding agents that could result in unintended clicks, remote code execution and software supply-chain actions.
That creates an uncomfortable crossover between data architecture and security. If information can influence the action, where that information came from, whether it can be trusted and how much weight the agent should give it are no longer abstract governance concerns. They become part of the execution model.
This is also why I would be careful with the idea that agents somehow make technical debt less important. An agent may make fragmented environments easier to navigate because it can reason about a task rather than rely entirely on a fixed integration path, but it still needs access to systems, usable interfaces, meaningful data, appropriate permissions and some idea of what to do when something fails halfway through.
If the API does not exist, the agent needs another way in. If the same customer has six identifiers, somebody or something still needs to establish whether they represent the same person. If an undocumented business rule matters to the decision, that knowledge needs to be available somewhere.
Putting an intelligent orchestration layer in front of a messy estate does not make the estate stop being messy. It might make the mess easier to navigate. It may also allow the consequences of that mess to propagate considerably faster.
Human oversight has an economic cost
At this point in most conversations somebody will say there will always be a human in the loop.
That sounds reassuring, but it raises an obvious question: if somebody still needs to review and approve everything the agent does, how much have we actually automated?
Imagine our customer service agent has checked five systems, identified an account problem, established that the customer was incorrectly charged and calculated that a £20 refund is appropriate. It then places the whole thing into a queue for somebody to click "Approve".
There may still be a meaningful saving because the investigation has been automated, but run that process across tens of thousands of transactions and you have retained a sizeable manual operation. The bottleneck has moved rather than disappeared.
This is where some of the safer language around agentic AI starts to conflict with the economics of automation. We introduce autonomy because we want software to remove work from a process, then become sufficiently nervous about the autonomy that we put a person back into every transaction.
Eventually that rather defeats the point.
The challenge with AI agent oversight is therefore that more oversight is not automatically better. Every approval step may reduce a particular type of risk, but it also reintroduces cost, latency and manual effort into the process. The better question is where human judgement genuinely adds value.
If the refund is £20, the rules are clear, all of the relevant systems agree and the action can easily be reversed, perhaps nobody needs to approve it. If the amount is £20,000, two systems disagree or the situation falls outside normal parameters, that sounds like an excellent time to involve someone.
For many processes, the useful destination is therefore probably not human in the loop but human by exception. Routine, well-understood work becomes increasingly autonomous, while unusual, ambiguous or high-consequence cases are escalated.
That still leaves plenty of room to introduce autonomy gradually. An agent can begin by observing and recommending. Once its behaviour is understood, it can prepare actions for approval. Eventually, specific classes of routine and reversible action can be executed automatically, with people becoming involved when the process crosses an agreed threshold.
OpenAI's guidance takes a similar risk-based approach. It recommends human intervention when agents exceed defined failure thresholds and around actions that are sensitive, irreversible or high stakes, such as large refunds or payments.
PocketOS arrived at the same issue from a considerably less theoretical direction. Following its database incident, destructive operations now require explicit human confirmation. The company did not respond by abandoning autonomous agents. It says it continues to run a number of narrow autonomous agents in production, while changing where the control boundary sits.
That seems to me the more useful way to think about human oversight. It is a control that should be applied where the consequence warrants it, not necessarily a permanent operating model for every action performed by autonomous software.
The point of automation is, after all, to automate. The design problem is deciding where people genuinely improve the outcome and where they are simply being retained because we do not yet trust the system enough to remove them.
Autonomy without traceability is operationally indefensible
If we move towards humans by exception, something else becomes more important rather than less: understanding what the agent actually did.
The Hugging Face investigation is a useful extreme example. Its forensic reconstruction covered roughly 17,600 actions across thousands of clusters, with investigators piecing together how the agent moved through systems and adapted its behaviour over time.
Bring that down to a normal business process. A customer says something changed incorrectly on their account and you discover an agent made the change.
"The AI did it" is unlikely to get you very far in the incident review.
You will want to know which agent acted, who initiated the task, which information it used, what that information contained at the time, which tools it called and which permission ultimately allowed the change. If another agent became involved along the way, that probably matters too.
This is where AI agent auditability stops being a governance nicety and becomes an operational requirement. The less direct human involvement there is in execution, the stronger the surrounding observability needs to become.
Microsoft's current guidance recommends logging the agent identity, its role and effective scope, the action performed, the affected resource and any user on whose behalf it was acting. NIST is likewise explicitly considering auditing and non-repudiation as part of the identity and authorisation problem for agents.
That does not mean every autonomous action needs somebody watching it. We already operate enormous automated environments without putting a person in front of every transaction, infrastructure event or network request. We establish limits, controls and monitoring, then investigate the things that fall outside them.
There is no obvious reason mature agentic systems should work differently.
Autonomy does not remove oversight. It changes when oversight happens.
Delegated intent complicates the trust boundary
There is one further development worth watching because it makes this problem more interesting again.
Agents are increasingly being designed to communicate and collaborate with other agents. Google's Agent2Agent protocol standardises how agents can discover and communicate with one another, including agents built by different teams and on different technology stacks. Its newer Agentic Resource Discovery work addresses the related question of how agents can find and verify tools, skills and other agents distributed across teams, organisations and platforms.
Imagine an internal agent receives a task and determines that another specialist agent can perform part of it. That second agent uses a tool which calls one of your services and ultimately changes something.
Who is acting?
The second agent? The first agent? The person who initiated the original task?
If the first agent is authorised to perform an action, can it delegate that authority? Does the second agent receive all of it, part of it or none of it? What happens if the agents belong to different organisations?
These are emerging questions and it would be premature to pretend the industry has settled on all of the answers. NIST's AI Agent Standards Initiative is explicitly conducting work around agent authentication and identity infrastructure to support secure human-agent and multi-agent interactions.
But the direction matters. Historically, trust has often been associated with a known user, application or integration. Once agents can dynamically discover capabilities and delegate parts of an objective elsewhere, trust and authority need to survive the journey.
The task can be dynamic. Accountability cannot disappear every time it crosses another boundary.
Agentic transformation should start with delegation, not technology
There is a pattern with significant technology shifts that most people involved in transformation will recognise. Something becomes strategically important, somebody decides the organisation needs it and everyone starts searching for somewhere to put it.
We need an AI strategy. We need some agent use cases. We should run a workshop. Which processes can we make agentic?
I would probably approach it the other way around.
Start with the process that annoys everybody. The one that is slow, expensive, fragmented or depends on somebody transferring information between systems all day. Understand where the decisions are made, what information those decisions rely on, which systems are involved, where the awkward exceptions occur and what happens when something goes wrong.
Then look at what the people in the process are actually doing. Are they exercising valuable judgement, or are they compensating for limitations in the technology around them? Are they approving something because there is genuine financial or operational risk, or because the workflow cannot distinguish a routine case from an unusual one? Are they manually reconciling information because the organisation genuinely needs their expertise, or because two systems use different identifiers?
Those are quite different problems.
Once you understand that, decide what you are prepared to delegate. Which decisions can software make? Which actions can it take? Which are reversible? Where do you need escalation? What authority does it require? What evidence would you need afterwards if somebody asked why a decision was made?
Only then does the technology choice become particularly interesting.
An agent may be an excellent answer. It may not. That is fine. Current guidance on agent design makes much the same distinction: agents are well suited to workflows where ambiguity, complex decisions or unstructured data cause deterministic approaches to struggle, but where those characteristics are absent, conventional automation may remain entirely sufficient.
The objective is not to deploy agents. It is to improve the business.
The architecture of automation is becoming the architecture of authority
For the last twenty years, digital transformation has largely involved connecting people to systems and systems to one another. We are now starting to introduce another participant into that estate: software that can receive an objective, determine how to pursue it and use those systems on our behalf.
There are some genuinely interesting possibilities in that, particularly around the coordination work that has resisted conventional automation. NIST itself describes the potential for agents to improve productivity, efficiency and decision making, while highlighting the need for appropriate identification and authorisation controls when those agents are given access to organisational data, tools and applications.
I think that combination is important. The opportunity and the risk come from the same characteristic: the agent has some freedom to decide how work gets done.
Once that happens, architecture is no longer concerned only with whether one system can connect to another. It increasingly determines which autonomous actor may use which information, exercise which capability, under whose authority, with what limits and with what evidence left behind.
In practice, enterprise AI agent governance is therefore unlikely to be solved by another policy document sitting somewhere on the intranet. It has to show up in the identities we create, the permissions we grant, the controls around consequential actions, the quality and provenance of the data agents consume, the escalation paths we design and the audit evidence we retain afterwards.
Hugging Face gives us an extreme example of autonomous software finding a route its operators did not expect. PocketOS gives us a much more everyday example of what happens when an agent simply has more authority than the task requires. The emerging work around agent identity, least privilege, secure multi-agent interaction and trusted resource discovery shows that the surrounding control model is already having to catch up with the capability.
None of that is an argument against agentic AI. Quite the opposite. If agents can remove some of the human coordination that makes modern digital businesses expensive and cumbersome to operate, there is a very tangible reason to pursue them.
But introducing autonomous actors into an existing digital estate is not simply another AI implementation. It changes assumptions about identity, trust, data, accountability and, ultimately, authority.
So before asking what an agent can do, I would start with a different question.
What are you actually prepared to let it do?
When AI Gets Authority
For the last few years, businesses have been concentrating on what generative AI can tell us. AI agents introduce a rather more consequential question: what happens when software can decide what to do next, and then actually do it?
In July 2026, an AI agent being used as part of an OpenAI cybersecurity evaluation compromised part of Hugging Face's production infrastructure. There is some important context here before we get carried away. This was not a normal business application suddenly deciding to attack another company. OpenAI was deliberately testing highly capable models against difficult cybersecurity tasks, and the production classifiers normally used to prevent models pursuing high-risk cyber activity were intentionally not enabled for the evaluation.
Even so, what happened is worth paying attention to. According to the subsequent investigations, the agent appears to have concluded that information held by Hugging Face could help it complete the benchmark it had been given. It found a way out through a zero-day vulnerability in infrastructure used by the evaluation environment, gained wider internet access, chained together other attack paths, obtained credentials and eventually reached Hugging Face systems. Hugging Face's forensic reconstruction covered around 17,600 individual actions, grouped into approximately 6,280 clusters.
The temptation, naturally, is to reach for the "AI escaped" or "AI went rogue" headline. I am not sure either is particularly useful. As far as the investigations could establish, the software was trying to achieve the objective it had been given and found a route nobody expected it to take. Hugging Face believes the agent was effectively trying to cheat the evaluation by reaching production systems where it thought benchmark information or solutions might be available.
That is considerably less theatrical, but from a digital transformation perspective probably more important.
For the last few years, most organisations have experienced generative AI as something they ask. Give it some information, ask a question and get something back. It might summarise a contract, analyse some data, write some code or draft an email, but there is generally still a person sitting on the other side deciding what happens next.
Agents start to blur that boundary. Give the same intelligence access to tools and it can retrieve information, call an API, update a record, execute some code, send the email or make the change itself. OpenAI describes agents in much the same way: systems that can independently manage workflow execution, make decisions, select appropriate tools and interact with external systems on a user's behalf.
The chatbot just got write access.
That shift is why AI agent governance is becoming a practical enterprise concern rather than simply another policy discussion. Once software can take action rather than make recommendations, questions about identity, permissions, controls, auditability and accountability move into the architecture itself.
Automation changes when software is given intent, not instructions
Software has been changing things for decades, so there is a danger of overselling what is new here. We do not need AI to update a database, trigger an API or run a workflow. Scripts, rules engines, orchestration platforms and automated processes have been doing those jobs perfectly well for years.
The difference is in how we get from the objective to the action. Traditional automation is normally designed around a route somebody has described in advance. When X happens, retrieve Y. If the answer is Z, perform this action. There may be hundreds of rules, branches and exceptions involved, but somebody has still tried to model the process beforehand.
Agents potentially allow us to move further towards giving software an outcome rather than every instruction required to reach it. Here is the problem. Work out what information you need. Decide which tools are appropriate. Deal with what comes back and get the job done.
That distinction matters because the execution path can become dynamic. An agent might start with one system, discover it needs some additional context, call something else, encounter a result it was not expecting, revise its approach and carry on. Current agent architectures are explicitly built around this ability to manage a workflow, choose tools and take actions depending on the state of the task.
That is a materially different proposition from conventional automation. The software is no longer simply executing a predetermined flow. Within whatever boundaries we establish around it, we are giving it some discretion over how that flow unfolds.
And that is where this gets much more interesting than another conversation about whether AI can write a better email.
The opportunity is to remove coordination, not people
A surprising amount of work inside an organisation is not difficult because any individual task is particularly complicated. It is difficult because of the coordination required between the tasks.
Open the CRM. Find the customer. Copy an identifier into the billing platform. Read the notes. Check another system. Work out what they mean. Ask another team whether something is normal. Wait for the answer. Update the first platform and send the customer an email.
Sound familiar?
We have spent years trying to automate those processes, and conventional workflow technology is extremely effective when the steps can be described predictably. The difficult bits tend to be the exceptions: the document that is not structured properly, the account that does not quite match, three systems that disagree with one another, or the point in the process where somebody needs to read something and understand what it means before they know what to do next.
Those are precisely the kinds of workflows where agents potentially become useful. OpenAI's current guidance specifically identifies complex decision making, difficult-to-maintain rules and heavy reliance on unstructured information as areas where an agentic approach may offer something that conventional deterministic automation does not. It also makes the important point that where those characteristics are absent, a deterministic solution may still be the better answer.
If software can take over some of the coordination between systems, information and decisions, there is potentially a meaningful efficiency gain. Not because everybody involved in the process suddenly disappears, but because we can stop using people as expensive middleware between platforms that were never particularly good at coordinating with each other.
There is an important distinction there. The objective should not necessarily be to remove people from the organisation. It should be to remove the work we currently make people do simply because the technology underneath them cannot handle context, ambiguity or exceptions very well.
That is why I think the opportunity around agents is much more significant than incremental productivity. Drafting a customer response ten seconds faster is useful. Allowing software to investigate the problem, reconcile the relevant information and carry the process forward is something else entirely.
Of course, the further it carries the process, the closer it eventually gets to an action with consequences.
And that is where things become rather more complicated.
Capability is not authority
A few months before the Hugging Face incident, PocketOS published a candid account of an AI agent deleting the company's production database. The deletion took nine seconds. Recovery took sixty hours. Its backups existed, but were three months out of date because the backup process had quietly stopped working.
PocketOS is also very clear that this was not simply an "AI problem". The company says the agent acted with confidence using permissions it should never have had, against an environment that did not have sufficient friction around the destructive action.
That is the useful lesson.
Humans have been deleting production databases for years, and software bugs have made a respectable contribution too. The interesting part is that the actor had sufficient authority for its mistake to matter.
We already understand this when the actor is human. Someone in customer services might be authorised to issue a £20 refund without anybody caring very much. £200 may still be entirely reasonable. £20,000 probably introduces another conversation.
The same applies elsewhere. An agent identifying a spelling mistake in some content metadata and correcting it is probably fairly benign. If the field it wants to change determines whether that content can be distributed in a particular territory, the consequence is very different. Likewise, an agent may correctly identify a production configuration issue, but that does not mean it should inherit unlimited rights to reconfigure the whole environment.
None of this is particularly exotic. It is identity and access management applied to a new type of actor, and I suspect it will become one of the foundations of effective agentic AI governance.
Microsoft is already treating agents in these terms. Its current guidance recommends a unique, dedicated AI agent identity, a named owner or sponsor, clearly documented purpose and data access, least-privilege access, controlled tool permissions, logging and tested revocation paths. NIST's work in the same area is explicitly examining identification, authorisation, auditing and non-repudiation for AI agents.
In practical terms, that means AI agent permissions need to be treated with the same seriousness as privileged human and service accounts. An agent can be perfectly capable of determining what it thinks should happen without being entitled to make it happen.
That distinction matters. AI agent controls cannot simply ask whether a model is technically capable of performing an action. They need to determine whether this particular agent, acting in this particular context, is authorised to perform it.
As an aside, I suspect AI agent sprawl will become one of those problems organisations do not think they have until somebody finally asks, "How many agents do we actually have?" We have been through variations of this with SaaS applications, cloud resources, service accounts and API keys already. There is no obvious reason the agent estate will organise itself any better, particularly in businesses where experimentation is deliberately devolved across teams.
That does not mean centralising every AI initiative. A central team is unlikely to understand every process well enough to identify all of the useful applications itself. Devolved innovation can be entirely sensible.
Devolved authority by accident is a different matter.
Agentic AI inherits every compromise in your digital estate
There is another awkward issue underneath all of this. Your agent is going to inherit your existing digital estate.
The shiny APIs, the legacy systems, the inconsistent customer records, the integration somebody wrote eight years ago and nobody wants to touch, and the platform that calls a customer one thing while another platform calls them something else. A company grows, platforms get bought, teams solve individual problems, acquisitions happen, suppliers change and priorities move. Eventually you end up with what everyone politely refers to as the current architecture.
Now imagine an agent dealing with a customer who has paid for a service but cannot access it. The answer might require information from CRM, billing, authentication, entitlement and support systems before anyone can determine what has actually happened.
A human operator will often deal with the inconsistencies almost without thinking about them. They know one system updates before another. They notice that the identifiers are slightly different. Perhaps they remember that customers on an old package behave differently, or simply ask the person sitting next to them because the documentation has not been updated since 2023.
Give the job to an agent and those assumptions need to exist somewhere. Which system is authoritative? What happens if billing says the payment succeeded but the entitlement platform says the account is inactive? Is the information wrong or is there a business rule the agent does not understand? Is one platform simply five minutes behind?
These are old data and architecture problems with a new consumer. The material difference is that the consumer may now take action on whatever conclusion it reaches.
Poor data feeding a chatbot can produce a bad answer. Poor data feeding autonomous software can produce a bad outcome.
There is another complication too. The information does not necessarily have to be wrong by accident. Research published in July demonstrated agent data injection attacks in which attacker-controlled information could be presented as apparently legitimate contextual data and influence what an agent subsequently did. The researchers identified vulnerabilities in real web and coding agents that could result in unintended clicks, remote code execution and software supply-chain actions.
That creates an uncomfortable crossover between data architecture and security. If information can influence the action, where that information came from, whether it can be trusted and how much weight the agent should give it are no longer abstract governance concerns. They become part of the execution model.
This is also why I would be careful with the idea that agents somehow make technical debt less important. An agent may make fragmented environments easier to navigate because it can reason about a task rather than rely entirely on a fixed integration path, but it still needs access to systems, usable interfaces, meaningful data, appropriate permissions and some idea of what to do when something fails halfway through.
If the API does not exist, the agent needs another way in. If the same customer has six identifiers, somebody or something still needs to establish whether they represent the same person. If an undocumented business rule matters to the decision, that knowledge needs to be available somewhere.
Putting an intelligent orchestration layer in front of a messy estate does not make the estate stop being messy. It might make the mess easier to navigate. It may also allow the consequences of that mess to propagate considerably faster.
Human oversight has an economic cost
At this point in most conversations somebody will say there will always be a human in the loop.
That sounds reassuring, but it raises an obvious question: if somebody still needs to review and approve everything the agent does, how much have we actually automated?
Imagine our customer service agent has checked five systems, identified an account problem, established that the customer was incorrectly charged and calculated that a £20 refund is appropriate. It then places the whole thing into a queue for somebody to click "Approve".
There may still be a meaningful saving because the investigation has been automated, but run that process across tens of thousands of transactions and you have retained a sizeable manual operation. The bottleneck has moved rather than disappeared.
This is where some of the safer language around agentic AI starts to conflict with the economics of automation. We introduce autonomy because we want software to remove work from a process, then become sufficiently nervous about the autonomy that we put a person back into every transaction.
Eventually that rather defeats the point.
The challenge with AI agent oversight is therefore that more oversight is not automatically better. Every approval step may reduce a particular type of risk, but it also reintroduces cost, latency and manual effort into the process. The better question is where human judgement genuinely adds value.
If the refund is £20, the rules are clear, all of the relevant systems agree and the action can easily be reversed, perhaps nobody needs to approve it. If the amount is £20,000, two systems disagree or the situation falls outside normal parameters, that sounds like an excellent time to involve someone.
For many processes, the useful destination is therefore probably not human in the loop but human by exception. Routine, well-understood work becomes increasingly autonomous, while unusual, ambiguous or high-consequence cases are escalated.
That still leaves plenty of room to introduce autonomy gradually. An agent can begin by observing and recommending. Once its behaviour is understood, it can prepare actions for approval. Eventually, specific classes of routine and reversible action can be executed automatically, with people becoming involved when the process crosses an agreed threshold.
OpenAI's guidance takes a similar risk-based approach. It recommends human intervention when agents exceed defined failure thresholds and around actions that are sensitive, irreversible or high stakes, such as large refunds or payments.
PocketOS arrived at the same issue from a considerably less theoretical direction. Following its database incident, destructive operations now require explicit human confirmation. The company did not respond by abandoning autonomous agents. It says it continues to run a number of narrow autonomous agents in production, while changing where the control boundary sits.
That seems to me the more useful way to think about human oversight. It is a control that should be applied where the consequence warrants it, not necessarily a permanent operating model for every action performed by autonomous software.
The point of automation is, after all, to automate. The design problem is deciding where people genuinely improve the outcome and where they are simply being retained because we do not yet trust the system enough to remove them.
Autonomy without traceability is operationally indefensible
If we move towards humans by exception, something else becomes more important rather than less: understanding what the agent actually did.
The Hugging Face investigation is a useful extreme example. Its forensic reconstruction covered roughly 17,600 actions across thousands of clusters, with investigators piecing together how the agent moved through systems and adapted its behaviour over time.
Bring that down to a normal business process. A customer says something changed incorrectly on their account and you discover an agent made the change.
"The AI did it" is unlikely to get you very far in the incident review.
You will want to know which agent acted, who initiated the task, which information it used, what that information contained at the time, which tools it called and which permission ultimately allowed the change. If another agent became involved along the way, that probably matters too.
This is where AI agent auditability stops being a governance nicety and becomes an operational requirement. The less direct human involvement there is in execution, the stronger the surrounding observability needs to become.
Microsoft's current guidance recommends logging the agent identity, its role and effective scope, the action performed, the affected resource and any user on whose behalf it was acting. NIST is likewise explicitly considering auditing and non-repudiation as part of the identity and authorisation problem for agents.
That does not mean every autonomous action needs somebody watching it. We already operate enormous automated environments without putting a person in front of every transaction, infrastructure event or network request. We establish limits, controls and monitoring, then investigate the things that fall outside them.
There is no obvious reason mature agentic systems should work differently.
Autonomy does not remove oversight. It changes when oversight happens.
Delegated intent complicates the trust boundary
There is one further development worth watching because it makes this problem more interesting again.
Agents are increasingly being designed to communicate and collaborate with other agents. Google's Agent2Agent protocol standardises how agents can discover and communicate with one another, including agents built by different teams and on different technology stacks. Its newer Agentic Resource Discovery work addresses the related question of how agents can find and verify tools, skills and other agents distributed across teams, organisations and platforms.
Imagine an internal agent receives a task and determines that another specialist agent can perform part of it. That second agent uses a tool which calls one of your services and ultimately changes something.
Who is acting?
The second agent? The first agent? The person who initiated the original task?
If the first agent is authorised to perform an action, can it delegate that authority? Does the second agent receive all of it, part of it or none of it? What happens if the agents belong to different organisations?
These are emerging questions and it would be premature to pretend the industry has settled on all of the answers. NIST's AI Agent Standards Initiative is explicitly conducting work around agent authentication and identity infrastructure to support secure human-agent and multi-agent interactions.
But the direction matters. Historically, trust has often been associated with a known user, application or integration. Once agents can dynamically discover capabilities and delegate parts of an objective elsewhere, trust and authority need to survive the journey.
The task can be dynamic. Accountability cannot disappear every time it crosses another boundary.
Agentic transformation should start with delegation, not technology
There is a pattern with significant technology shifts that most people involved in transformation will recognise. Something becomes strategically important, somebody decides the organisation needs it and everyone starts searching for somewhere to put it.
We need an AI strategy. We need some agent use cases. We should run a workshop. Which processes can we make agentic?
I would probably approach it the other way around.
Start with the process that annoys everybody. The one that is slow, expensive, fragmented or depends on somebody transferring information between systems all day. Understand where the decisions are made, what information those decisions rely on, which systems are involved, where the awkward exceptions occur and what happens when something goes wrong.
Then look at what the people in the process are actually doing. Are they exercising valuable judgement, or are they compensating for limitations in the technology around them? Are they approving something because there is genuine financial or operational risk, or because the workflow cannot distinguish a routine case from an unusual one? Are they manually reconciling information because the organisation genuinely needs their expertise, or because two systems use different identifiers?
Those are quite different problems.
Once you understand that, decide what you are prepared to delegate. Which decisions can software make? Which actions can it take? Which are reversible? Where do you need escalation? What authority does it require? What evidence would you need afterwards if somebody asked why a decision was made?
Only then does the technology choice become particularly interesting.
An agent may be an excellent answer. It may not. That is fine. Current guidance on agent design makes much the same distinction: agents are well suited to workflows where ambiguity, complex decisions or unstructured data cause deterministic approaches to struggle, but where those characteristics are absent, conventional automation may remain entirely sufficient.
The objective is not to deploy agents. It is to improve the business.
The architecture of automation is becoming the architecture of authority
For the last twenty years, digital transformation has largely involved connecting people to systems and systems to one another. We are now starting to introduce another participant into that estate: software that can receive an objective, determine how to pursue it and use those systems on our behalf.
There are some genuinely interesting possibilities in that, particularly around the coordination work that has resisted conventional automation. NIST itself describes the potential for agents to improve productivity, efficiency and decision making, while highlighting the need for appropriate identification and authorisation controls when those agents are given access to organisational data, tools and applications.
I think that combination is important. The opportunity and the risk come from the same characteristic: the agent has some freedom to decide how work gets done.
Once that happens, architecture is no longer concerned only with whether one system can connect to another. It increasingly determines which autonomous actor may use which information, exercise which capability, under whose authority, with what limits and with what evidence left behind.
In practice, enterprise AI agent governance is therefore unlikely to be solved by another policy document sitting somewhere on the intranet. It has to show up in the identities we create, the permissions we grant, the controls around consequential actions, the quality and provenance of the data agents consume, the escalation paths we design and the audit evidence we retain afterwards.
Hugging Face gives us an extreme example of autonomous software finding a route its operators did not expect. PocketOS gives us a much more everyday example of what happens when an agent simply has more authority than the task requires. The emerging work around agent identity, least privilege, secure multi-agent interaction and trusted resource discovery shows that the surrounding control model is already having to catch up with the capability.
None of that is an argument against agentic AI. Quite the opposite. If agents can remove some of the human coordination that makes modern digital businesses expensive and cumbersome to operate, there is a very tangible reason to pursue them.
But introducing autonomous actors into an existing digital estate is not simply another AI implementation. It changes assumptions about identity, trust, data, accountability and, ultimately, authority.
So before asking what an agent can do, I would start with a different question.
What are you actually prepared to let it do?
If the challenges discussed here feel familiar, or you’re thinking about how agentic AI fits into your wider digital estate, we’re always happy to compare notes. Drop us a line at hello@spicymango.co.uk, give us a call, or get in touch through our contact form.
If the challenges discussed here feel familiar, or you’re thinking about how agentic AI fits into your wider digital estate, we’re always happy to compare notes. Drop us a line at hello@spicymango.co.uk, give us a call, or get in touch through our contact form.
If the challenges discussed here feel familiar, or you’re thinking about how agentic AI fits into your wider digital estate, we’re always happy to compare notes. Drop us a line at hello@spicymango.co.uk, give us a call, or get in touch through our contact form.
