BLUE-LAB — home
, , , ,

AI-Enabled Exfiltration: getting Tricky.

Neon line-art of four bracketed tiles (a magenta poisoned email and web page with a hidden fish hook, a plug connector with a green approved light, a database with a key, a git repository) linked by circuit traces into a glowing yellow double-walled hexagon holding a crystalline AI agent core with orbiting wrench, plug and key glyphs, while magenta file and key fragments escape through a breach at the top

Ok so MITRE added a new one in ATT&CK v20, T1700 AI-Enabled Exfiltration. In short, an attacker gets instructions in front of your AI agent, and the agent uses the tools you gave it to send your data somewhere it should not go.

Note, nothing here is a new exfil channel. The agent already has access, connectors, and usually a token, secret, credential, etc. which gives privilege and entitlement. Once an agent is coerced to exfiltrate, the transfer itself usually lands on techniques we already know e.g. Exfiltration Over Web Service (T1567) or Exfiltration to Code Repository (T1567.001). In the logs it can look like the agent just doing its job… the old normally abnormal in normal!

Ok so, Indirect Prompt Injection (T1698), instructions hidden in content the model processes, say a web page, an email, a ticket or a document in a RAG (Retrieval-Augmented Generation) store. The user never types anything malicious, the agent reads it while doing a normal task.

Direct Prompt Injection (T1699) is the version where the instructions are typed straight into the model.

The simplest way to explain the risk is the “lethal trifecta”. An agent that can read private data, is exposed to untrusted content, and can communicate with unknown and untrusted systems… this last bit is the kicker! The agent has everything an attacker needs. Take any one of those away and this technique gets a lot harder.

There is a load of info in the blog post OWASP quick fire round, T1700 is basically where those two meet.

So what does this look like for real? Mandiant had a good example, their offensive security team convinced a client’s internal assistant, one that helps developers with their build and release pipelines, that it was a security researcher on an authorised test. They then gave it a GitHub PAT for a repo they controlled, and because GitHub was an allowed domain, the agent cloned internal repositories and pushed them out using its own tools. Essentially the agent just doing its job, for someone else.

It is not just red teams. Anthropic saw an attacker inject instructions into an AI vendor’s testing sandbox and walk off with the vendor’s AI API keys, and Barracuda are seeing phishing emails with hidden instructions aimed at the AI assistant, not just the person reading it. Google expect it to keep growing on the public web too.

Hunting for Bad

MITRE have a set of detections for this, DET0927, with three analytics.

To help understand this table, specifically the “What you have to define first” column consider it as a list of blanks or parameters, but these parameters are literally for you to fill in as a human. Each name is a blank you fill in. For example, AuthorizedAIEndpoints is the blank, and you could fill it in with “Azure OpenAI and SharePoint”. The detection then flags it when the whole chain steps outside what you wrote in, a single mismatch on its own will not fire it.

AnalyticPlatformLog sourcesWhat you have to define first (MITRE Mutable Elements)
AN2078SaaSSaaS audit log of the actual send, share, upload or write. AWS Bedrock AgentTrace (action group and knowledge base inputs)ProtectedDataScope, AuthorizedAgentActions, AuthorizedAIDataFlows, AuthorizedAIOutputDestinations, AIExfiltrationWindow
AN2079WindowsSecurity 4656, 4663, 4670 (file access). PowerShell 4103 to 4106. Sysmon 1AuthorizedAIDataScope, ExpectedAIProcessTree, AuthorizedAIEndpoints, EndpointExfilWindow
AN2080Linuxauditd open/read, EXECVE and connectSame as AN2079

Note, the technique page only lists SaaS and Windows, but there is a Linux analytic too, so do not drop your Linux agent hosts out of scope.

WTF are Mutable Elements?!

Also I might hear you ask “what are MITRE Mutable Elements?!”

Well here is an example of how they are used

An analyst observes an AI-associated process access data outside $AuthorizedAIDataScope$, followed within $EndpointExfilWindow$ by transfer-capable execution and an outbound connection to a destination outside $AuthorizedAIEndpoints$ from the same process lineage.

What it looks like when you fill it in:
An analyst observes an AI-associated process access data outside the synced Sales folder on the agent host (C:\Agents\Sales), followed within 5 minutes by transfer-capable execution and an outbound connection to a destination outside our Azure OpenAI service from the same process lineage.

How to read it: “If our AI touches anything other than Sales files, and within 5 minutes that same process, or something it started, runs something that can send data and connects anywhere other than our Azure OpenAI, that’s suspicious.”

“Same process lineage” means

  • The read
  • The send
  • The connection

They all trace back to the same process and the things it started, so two unrelated events on the box do not get stitched together. And “AI-associated process” is a blank too, $ExpectedAIProcessTree$, the normal parent and child processes your agent runs as, say the agent service starting python.exe, taken from clean runs.

I thought it best to call that out as I don’t want to assume this syntactic read of those parameters makes sense. Can you spot a pain point I had :D.

…Back to the Article 😀

So if nobody has recorded what an agent is allowed to read and where it is allowed to send, there is nothing to compare against, and that matters more now than ever. Do not wait for an incident to find out nobody wrote it down, this is an identity and data governance job before it is a SIEM job.

you could see the detection logic as:

WHEN an AI agent identity or AI-associated process reads data outside its approved scope
AND within the normal access-to-transfer window the same identity or process lineage calls a transfer-capable tool or API
AND the destination (recipient, repo, tenant, domain or connector) is not on its approved list
THEN alert and pull the prompt and tool-call history for that agent session

A tool call on its own shows an attempt, not proof that data actually left, for that you want the destination’s own record of the transfer.

Things to look for

Mandiant mention first time access to sensitive repositories by an agent as a strong signal (at least one worth eyeing over). Agents are noisy, but they are also repetitive or at least they should be if following a least privileged approach with Agents built for a specific purpose, so anything new is interesting.

It is not a complete rule though, a new project or integration will look just the same at first. Same goes for an AI service account opening a new outbound connection, or pushing to a remote outside your organisation.

I would watch where things are going more than what is in them to get the initial feel. Anything that has to spot the secret in the outbound data can be beaten by encoding or ciphering it first. Tool calls with long encoded strings or odd “signature” or config fields are still worth a look, just do not rely on that alone. Systems that are designed to analyse this type of activity data will certainly help with this.

Can we stop it?

MITRE’s mitigations are Application Isolation and Sandboxing (M1048), Restrict File and Directory Permissions (M1022) and User Training (M1017), which is fine as a start.

Why not just tell the agent not to leak? A research preprint MITRE cite on T1700, The Framing Gap (Rahman and Kim, 2026), tested that. Their agent was asked to summarise a web page while holding a fake secret it was told never to send anywhere but one trusted host. The attacker could only change that page, so every attack was written into the page itself. The obvious injections, ten styles including hidden HTML, a poisoned search result, a base64-encoded payload and plain social engineering, got nowhere with GPT-4o, 0%.

Same page, same leak, but reword the request so it reads like part of the task, say “add an integrity signature to your summary”, a config field to fill in, or a trusted-looking host to send it to, and the strongest wordings worked every time. Think of a phishing test, staff who would never reply to “send me your password” will happily follow “as part of month end, add the account code to the invoice reference”. The model was “tricked” into doing something it wasn’t designed to do.

What held was the structural stuff, a fixed list of where the agent is allowed to send data, and splitting the agent so the part reading untrusted content never holds the secrets. Just make sure that list is the actual repo, tenant or recipient and not just the domain, GitHub being allowed is exactly how the Mandiant test got out.

I liken this to * firewall rules, or *’s in server certificates, or service certs.

An agent is just another entity with an identity. If one holds a secret to a repo and has a route to the internet, I want to know who owns it, why it needs both, and when somebody last checked. Essentially Life Cycle Management principles.

The Principle of Least Privilege (PoLP) plays a role here, the initial access to the agent for instance is a control point that was missed. The agent could ask for authentication and authorisation before actions are started. Heck even Separation of Duty or S o D makes sense.

If it does happen

The attacker was steering the agent, not holding its password, so a password reset alone does little. Disabling the account stops the next sign-in, but sessions and tokens already issued carry on until they expire or you revoke them.

  1. Cut the agent’s route out and revoke its live sessions, but keep it running so the prompt history is there to investigate (Mandiant’s advice).
  2. Strip the rights. Take the agent out of every access policy, whether it is named in the policy directly or gets in through a group the policy allows. Either route is enough to reach a server, and a direct entry is easy to miss in a group-based review. Make the change where your provisioning reads from, or the next sync puts it back.
  3. Kill whatever re-dumps, the poisoned instruction in the agent’s memory or index, and any workflow, webhook, scheduled task or forwarding rule that was created. Otherwise the next run just collects whatever you rotate.
  4. Then rotate everything that left, keys, tokens and certificates as well as passwords. Change them directly, or through a vault that takes the account back if anyone changes it outside the vault. With the agent out of the policy, a stolen or re-dumped credential no longer opens that server.
  5. Find the poisoned page, email or ticket, or the next run does it all again.

So … lets wrap this one up.

T1700 is not a new way out. It is an identity with access, doing its job for someone else, and telling it the rules did not stop a well-worded request. What works is the identity tooling we already have, pointed at a new kind of entity.

Before anything happens, find them. You cannot govern an agent you do not know about, so start with every agent, the identities it holds and the secrets it uses. Then put each one into your identity governance (Identity Governance and Administration, IGA) like any other identity. Give it an owner, record what it may read and where it may send (the blanks MITRE left you), and review it like a service account. Keep its secrets in a vault (Privileged Access Management, PAM), checked out when needed or injected at connection time, so there is no secret sitting in the agent to leak.

When it does go wrong, the same tools are your incident response kit. Identity Threat Detection and Response (ITDR) can make the first cut, revoking sessions and blocking the agent while you investigate. IGA tells you what the agent had and who owns it, and pulls that access across the apps it is connected to in one go.

Clear out whatever may dump credentials again, maybe even rebuild, rotate the secrets in an automated way where the process is auditable and evident. Two logs tell you what could have left: the logon events show where the agent’s credentials were used, and the vault’s retrieval log shows which stored credentials it checked out.

The attacker only needed to get words in front of the agent, and its scope did the rest. The model behind it may well get better at spotting the tactics and techniques, but what an agent can reach is tied to identity, networks, and policy, and you can make the call on what that should be today 🙂 At least you should discover them.

This was a longer one


Sources and further reading

The technique

Research and threat intelligence

Identity governance

Containment