Nobody Cracked the Vault. They Talked the Teller Into Opening It.

I read this cybersecurity advisory about a week ago from CISA (the Cybersecurity and Infrastructure Security Agency), issued jointly with the NSA and FBI. Here's a quick summary. The full advisory is linked in the sources below.

On September 8, the three agencies published AA26-251A, naming six China-based AI companies, DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI, for running industrial-scale distillation campaigns against U.S. frontier models: Claude, GPT, Gemini, and Grok. Since late 2024. Billions of tokens across millions of requests. The agencies say it likely happened with Chinese government awareness, and that it wasn't a side project. It was the core of how these companies built their models.

Here's what makes this different from every other breach story. Nobody broke into anything. They bought API access through shared premium accounts and a gray market of proxies called "transfer stations," and asked the models millions of questions designed to make them show their reasoning. Then they trained their own models on the answers.

Think of it like a bank. Nobody cracked the vault. They walked up to the teller, asked the right questions in the right order, and the teller handed it over, because answering questions is the teller's job.

That's the attack. It walks in the front door with a paid subscription.

CISSP: same eight domains, new kind of asset

When I studied for the CISSP, Asset Security meant files, databases, source code. The stuff in the vault. This advisory stretches that. The asset here isn't the model weights on a disk. It's the model's behavior: the chain-of-thought reasoning, the tool-use patterns. That's the intellectual property, and it leaves the building one inference call at a time, through the teller window.

And from the IAM side, the line I keep coming back to: paying for an API key does not mean you're authorized for industrial cloning. Having a bank account doesn't entitle you to the vault. CISA's indicators are all identity and behavior. Shared accounts from multiple IPs. New subscriptions instantly at max throughput. 24/7 usage with no human idle pattern. That's not a firewall problem. That's accountability and behavioral analytics applied to API keys instead of employee logins.

It's already in the OWASP list

I wrote about the 2026 OWASP GenAI LLM Top 10 a couple of weeks ago. Look at this advisory through that list, and it's almost all there:

  • Jailbreaking the model into dumping its hidden reasoning - LLM01 - Prompt Injection.

  • Model revealing restricted output: reasoning traces, internal behavior - LLM02 Sensitive Information Disclosure.

  • Automated agents running collection at scale, unchecked - LLM03 Excessive Agency.

  • 24/7 max-throughput usage tied to a single account - LLM06 Unbounded Consumption.

Number one and number two on the list, plus the two that just moved up. Understand the OWASP list before you deploy an agent or put a model on your site. A government advisory just described those concepts as a nation-state playbook.

The controls that actually work

Nothing stops a funded, adaptive collection program outright. What you can do is change the economics: more accounts burned, worse data extracted, faster detection. In bank terms, you can't stop people from asking the teller questions. You can train the teller, limit what's behind the window, and notice when the same customer has been at the counter for three days straight.

Identity and consumption. Real identity proofing. Break the shared-subscription game. When an account runs at 100% around the clock, dig in. What is it actually doing?

Understand your inference API as an attack surface. MITRE ATLAS calls it inference API access, and CISA mapped this whole campaign to it. Information gets gathered one question at a time and pieced together, which is why the next control matters.

Differential privacy and output controls. CISA points to NIST's adversarial ML guidance here. Perturb outputs, withhold reasoning traces, degrade answers for high-confidence distillation traffic. Clean up what goes out so no single answer is the complete picture.

Checkpoints before output. A second agent reviewing the first agent's output before it reaches the user. Multiple checkpoints so what leaves the system has been cleared. It shrinks the blast radius. The honest caveat: a checker agent can be fooled too. It's a layer, not a wall. But layers are the point.

AAISM: this is the job

The AAISM's three domains map onto this advisory almost line by line. Governance: write into policy that distillation at scale is a security incident, not "heavy usage." Risk management: transfer stations and aggregators are vendors in your inference path. Assess them, contract them, or cut them. Controls: authenticated APIs, per-key quotas, output filtering, monitoring for the exact indicators CISA listed.

And the one CISA recommendation that makes governance people uncomfortable: silently serve worse answers to suspected distillers and never tell them. That's deception applied to model outputs, and it only works if governance pre-authorized it, because ops can't call a lawyer at 2 a.m. for permission to degrade a model. That decision belongs in the playbook before the incident.

The part for everyone who isn't a frontier lab

Look at CISA's indicators again: constant usage, instant max throughput, no idle time. Now look at your own production agent fleet running against a frontier API on one shared key. You fit the profile. Per-agent identities, no shared keys, and know what your traffic looks like from your provider's side.

There are nation-state actors whose job is to extract intellectual property from these systems to make their side stronger. That's not paranoia. It's a joint advisory from three agencies. Staying current on those notices is part of the job now.

Next up: MAESTRO

One framework that would have made this attack visible on paper before it happened is MAESTRO, the Cloud Security Alliance's threat-modeling method for agentic AI. Seven layers, from the foundation model to the agent ecosystem. It's a map, not a control. You don't install it, you think with it. I don't see many people using it yet, so my next post is a walkthrough of what it is and how to actually run one.

The advisory's attributions are the agencies' assessments; nothing here is legal advice.

Sources

  • NSA/CISA/FBI: AA26-251A: China-Based AI Companies Conducting Industrial-Scale Distillation Campaigns Against U.S. AI Companies (Sept 8, 2026): cisa.gov/news-events/cybersecurity-advisories/aa26-251a

  • NIST: AI 100-2 E2025: Adversarial Machine Learning: nvlpubs.nist.gov

  • MITRE ATLAS: atlas.mitre.org

  • OWASP GenAI Security Project: LLM Top 10 2026: genai.owasp.org

  • Anthropic: Detecting and preventing distillation attacks (linked from the advisory)

  • My earlier post: OWASP Admitted the Model Will Be Fooled