If your company already has an assistant drafting replies to customers, a workflow that classifies invoices or an agent that queries ERP data, AI agent security has stopped being a technical debate and become a management decision. A recently circulated report on the offensive capabilities of large models points to a conclusion worth understanding properly: the risk lies less in the model you choose than in what you let it touch. And that, for a mid-sized company, is good news, because it means there is real room for control.
What the index measured and why 15th place matters more than first
According to the published information, Booz Allen Hamilton evaluated 18 frontier models —nine American and nine Chinese— measuring two things: their ability to find software vulnerabilities without access to the source code and their ability to complete a real intrusion against a corporate network with active defence. The interesting methodological detail is that what was scored was what the attacked systems recorded, not what the model claimed to have achieved.
The headline was that an Anthropic model came first with 80 points out of 100, far ahead of the second (49). But the relevant finding is further down the table: a model that on its own had scored barely 13 points reached the same 80 when equipped with an attack harness, that is, with the software that connects it to tools and keeps the task in focus. Translated into business language: the difference between 1st place and 15th place was not the brain, it was the plumbing.
There is a second data point from the same report worth keeping in mind. One model refused to complete a task for lack of credentials, and a variant of the same model fine-tuned for cybersecurity tasks executed it without any problem. In other words: the safeguard was not an immutable property of the model, it was a consequence of how it was configured.
It is worth adding a note of honesty: the firm itself published a product to curb automated attacks on the same day, which invites reading its recommendations with a certain scepticism. The data from the exercise, on the other hand, is perfectly useful to think with.
AI agent security does not live in the model, it lives in the integration
Here is the reading that matters to us for a private company that is automating processes. When we talk about an AI agent we are not talking about a chat that answers questions, but about a system that executes multi-step tasks using tools: reading an email, querying the ERP, creating an order, sending a file. That set of connections —the tools, the permissions, the credentials, the system instructions, the degree of autonomy— is exactly what the report calls a harness. And that is where the level of risk is decided.
Put another way: the same setup that turns a mediocre model into a competent attacker is the one that turns any model into a useful agent for your operation. It is the same piece. That is why, when we assess an AI project with a client, the interesting conversation is not «which model do we use», but «what can this agent do, with what permissions and who supervises it».
Four controls you can review this week
You don’t need a security department to get started. You need order.
1. Inventory of agents, tools and permissions
Write down in a single table what assistants or automations are running in the company, who set them up, which systems they connect to and with which user. It is common to find workflows created by a department with administrator permissions «because that’s how it worked». Apply least privilege: each agent accesses only the data and actions it needs for its task.
2. Graduated autonomy and human confirmation
Define which actions an agent can chain together without anyone validating them: reading and classifying, yes; issuing a payment, contacting a customer or modifying an item master, with confirmation. The greater the impact of an action, the closer a person should be.
3. Instructions and restrictions treated as code
If the safeguards live in the configuration, they must be managed the way software is managed: versioned, documented, reviewed when you change model or provider, and tested before going into production. An agent that was working well may behave differently after a model update.
4. Traceability and governed data
You need records of what the agent did, with which tool and at what moment; first to debug errors and then to be able to respond to a customer, an auditor or a future regulatory requirement. And you need orderly data: without data governance —quality, access, privacy— neither does AI deliver nor can the risk be contained.
How we bring this down to earth at Tisa
At Tisa we have been implementing management systems since 1987 and, from our AI and data practice, we help companies take realistic steps: identifying the relevant data, creating data governance protocols aligned with AI regulation, assessing tools according to investment capacity and, above all, developing and executing a concrete use case adapted to the business. We also work on the basic cybersecurity side —risk assessment and response plans— and on automation with Power Platform and Microsoft Dynamics 365 Business Central.
That approach is what makes us insist on an uncomfortable but useful message: AI adds value when it solves a measurable problem and when it is deployed with permissions, supervision and records thought through from the start. The rush to «have AI» is precisely what creates holes.
From the headline to the work plan
If this index makes anything clear, it is that the capability bar is rising fast and that the competitive advantage —and the risk— lie in the integration, not in the model’s brand. The company that today documents its automations, tidies up its permissions and chooses its first use case well will arrive far better prepared for whatever comes, in technology and in regulation.
Are you rolling out automations or AI assistants and want to review the design before scaling them? Let’s talk with no obligation: call us on (+34) 971 305 885, write to us at info@grupotisa.com or visit grupotisa.com. We analyse your situation and tell you frankly where to start.
*Source of the topic: article «Claude Mythos leads Booz Allen’s Cyber Weapon Index», El Ecosistema Startup (
