Anthropic said it has strengthened defenses against attempts to extract large amounts of information about the reasoning process of its conversational AI, Claude—known as reasoning tokens—and reduced the volume extracted by 90% to 99%. The company said it has restricted access to reasoning data, deployed classifiers to detect signs of extraction, and introduced a system that automatically suspends accounts flagged as suspicious. It says tens of thousands of accounts have been suspended so far.
How to stop large-scale access to Claude
The practice at the heart of the issue is “distillation,” a technique in which the outputs of a powerful AI model are used as training material for a smaller, separate model. It is a legitimate way to make a company’s own models more lightweight, but it can also be used to send large numbers of queries to another company’s model and use the responses to copy its capabilities wholesale. Anthropic is concerned about organized extraction of the latter kind.
According to the company, attackers are using networks of hundreds of thousands of fake accounts. Each account is kept to a level of activity that looks like an ordinary user, while the network as a whole gathers a huge volume of responses. It is a tactic one might call an “account swarm” rather than a human wave.
What the attackers are after are reasoning tokens—in other words, the internal steps a model takes on the way to an answer. If they can obtain not just the final answer but also the path of reasoning behind it, they can teach another model “how to think.” To use a cooking analogy, it is like taking not only the finished dish to imitate its taste, but the recipe and instructions as well.
Anthropic’s countermeasures fall into three broad categories: restricting access to Claude’s internal reasoning steps, running classifiers to detect extraction attempts, and automatically suspending accounts deemed suspicious.
What the numbers say about the measures’ impact—and their limits
Anthropic says the volume of reasoning tokens extracted fell by 90% to 99% after its defenses went into operation. It also says the number of successful distillation efforts dropped sharply after the measures were introduced. The range is broad, but the company’s claim is that extraction was brought down to one-tenth of its previous level or less.
Another important point is the side effect Anthropic itself acknowledges: legitimate users can sometimes be caught by the detection systems. The tighter the net for filtering out abuse, the more likely it is to catch ordinary users and businesses as well. This familiar trade-off in security measures is no exception for AI services.
Developers who make heavy use of the service, and business systems with automation tools built in, may exhibit behavior that resembles signs of extraction. Going forward, the balance between the strength of anti-abuse measures and operating the service without disrupting normal use will affect how easy it is to use.
What the accusations naming companies mean
Anthropic named Chinese companies Zhipu (智譜) and Moonshot as parties involved. According to the company, Zhipu elicited cyber-related capabilities from leading U.S. models and used them to benefit its own GLM-5.3. Anthropic also alleges that Moonshot secretly routed queries through Claude. These are Anthropic’s claims, and the companies’ views could not be confirmed in the materials released this time.
Anthropic also raised the possibility that capabilities extracted this way could end up in foreign military and surveillance systems. The company’s framing of the issue as a matter of national security, rather than merely commercial imitation, stands out.
In another development, the company has urged other AI developers to adopt similar defenses. Securing the gates at just one company would have limited value if the information could simply be extracted through another entry point. The question is whether the industry can coordinate its efforts.
The generative AI race is expanding beyond the contest to build smarter models into a contest over identifying and controlling who uses those models and how. Unflashy defensive technologies, such as detecting fake accounts and restricting access, are likely to play an increasingly important role in determining AI companies’ competitiveness.
Comments