How (some) Chinese AI Practitioners View Model Distillation

A LatePost piece: The Distillation Storm: The Technical Race AI Companies Don’t Want to Discuss in Public

This is a translation of a piece by LatePost (晚点), a Chinese tech media outlet known for its close sourcing within China’s internet and AI industry. The article draws on interviews with nearly ten researchers and practitioners at various Chinese AI companies, combined with public research and technical reports, to reconstruct the history and current state of model distillation — what it actually is, how industrial-scale distillation works, and whether it can be a lasting competitive advantage.

The Distillation Storm: The Technical Race AI Companies Don’t Want to Discuss in Public

Original by the LatePost Team (晚点团队)
August 13, 2026

By Cheng Manqi (程曼祺) | Edited by Song Wei (宋玮)

No one wants to talk about it publicly, but everyone is quietly watching it.

Some see it as an unseemly act of theft. Others believe it has been stigmatized by a small number of leading companies for their own benefit, and that in itself it is merely an optimization method.

Over the past several months, many loose threads in the AI field have pointed to the same node: distillation.

The changes and events connected to it include open-source models closing in on the strongest closed-source models; 77 U.S. companies signing an open letter opposing rushed restrictions on open-source models; Anthropic twice accusing Chinese companies of using large numbers of fraudulent accounts to extract data; a $1.5 billion copyright settlement; and ByteDance’s founder’s direct response at a ByteDance Seed all-hands meeting about “not distilling.”

This technology, which has existed for many years, has been repeatedly mentioned, discussed, misunderstood, and distorted in 2026. What exactly is distillation? How is large-scale distillation carried out? Can distillation become a moat for a model development team? And what is its cost?

We interviewed nearly ten researchers and practitioners in the model field from different companies, and combined those interviews with public research and technical reports, to reconstruct the past and present of distillation, as well as the further changes it is bringing.

The starting point of distillation: compression, not getting stronger

Distillation is not plagiarism. It is not stealing software code, and it cannot directly obtain another model’s weights or its complete training data.

The kind of distillation now at the center of controversy — that is, distillation that makes a model stronger — is technically a way to obtain high-quality data: repeatedly ask a stronger “teacher model” questions, get answers, and then use these question-answer data pairs to train another “student model,” so that the latter reaches similar performance.

The idea of distillation has been around for a long time. In 2015, Geoffrey Hinton, who had recently joined Google Brain, published “Distilling the Knowledge in a Neural Network” together with Jeff Dean, then head of Google Brain, and the young researcher Oriol Vinyals. It was the first time earlier ideas such as model compression, which had appeared in 2006, were summarized under the term “distillation.”

At that time, Google was still nearly two and a half years away from proposing the Transformer architecture, which is now the foundation of large language models. Hinton and his co-authors applied the idea of distillation to image recognition models. Their method was to have the student model learn the probability distribution output by the teacher model. For example, recognizing that a cat is a cat is, for deep learning, a statistical process: 0.7 cat, 0.2 fox, 0.1 dog → cat.

The student model can see this set of probability distributions output by the teacher. This is learning the “logits,” in what is called “soft distillation.” Logits are a set of raw scores; after Softmax conversion, they become a probability distribution.

Soft distillation is usually “white-box distillation,” because it requires the teacher model’s output probabilities to be fully open to the student model.

This kind of distillation usually happens within the same organization. Its purpose is not to make a model stronger, but to “compress” it — to use a smaller-parameter model to approximate the capabilities of a larger-parameter model. Some performance is lost, but inference becomes faster and cheaper.

Even today, “compression” remains one of the most typical uses of distillation. In autonomous driving, for example, companies first build a stronger cloud-based large model, and then use methods such as distillation and pruning to turn it into a smaller model that can run on in-car chips. Companies including Li Auto (理想) and XPeng (小鹏) have such practices.

A more recent example is DeepSeek-R1 in early 2025. At that time, DeepSeek also released six small distilled models. Their teacher model was R1 itself, with 671 billion total parameters. Among the student models, four used Alibaba’s Qwen2.5 as the base, and two used Meta’s Llama 3 as the base. The smallest had 1.5 billion parameters, and the largest had 70 billion.

DeepSeek first had R1 generate about 600,000 pieces of reasoning data in the format of “question–reasoning process–answer,” along with about 200,000 pieces of non-reasoning data. It then used these data in the post-training stage to supervised fine-tune six small base models. All of these models gained stronger reasoning abilities.

LatePost previously reported that after the 2026 Spring Festival, Guo Daya (郭达雅), one of the core authors of R1, joined ByteDance Seed.

“Industrial-scale distillation attacks”

When used for compression, distillation is a neutral technical method. But by February 2026, Google and Anthropic had both published articles that used “distillation” together with “attack,” directly accusing some companies of using distillation for unfair competition.

Google viewed this as a form of IP theft:

Over the past year, “distillation attacks” have increasingly been used as a means of intellectual property theft.
— GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use

Anthropic said in February and June that DeepSeek, Moonshot AI (月之暗面, Kimi), MiniMax, and Alibaba Qwen (阿里千问) had used roughly 50,000 fraudulent accounts to conduct more than 44.8 million interactions with Claude, attempting to extract Claude’s capabilities. The February document was titled “Detecting and preventing distillation attacks”; the June claim appeared in Anthropic’s letter to the U.S. Senate. The companies mentioned did not respond directly.

How did distillation of leading closed-source models come to be carried out at large scale? The source of the change is still technical. There are three lines of development.

From soft distillation to hard distillation, from white-box to black-box

In 2016, Yoon Kim, then a PhD student at Harvard, and his adviser Sasha Rush proposed sequence-level knowledge distillation, applying the method that had first been used in image recognition to the language task of translation.

Sequence-level distillation no longer learns the probability distribution output by the model at each step. Instead, the teacher model first generates high-quality translations, and then the student model learns the complete “source text–translation” sequence pairs.

This technique was originally proposed to compress large translation models and improve decoding speed. But it also had another effect: distillation no longer needed to know the teacher model’s step-by-step probability distribution. It could be done by looking only at the “final answer.”

This is “hard distillation.” It can be done in a black-box way, meaning that one only needs to call an API and directly obtain the teacher model’s answer.

Sasha Rush later joined the AI programming company Cursor in March 2025. In June this year, Cursor was acquired by SpaceXAI — the new name after the merger of SpaceX and xAI — for $60 billion, and the team has already been integrated.

The rise of reasoning models

In September 2024, just as discussion was spreading about the Scaling Law hitting a wall, OpenAI released the o1 reasoning model. o1 brought two changes.

Large-scale reinforcement learning in the post-training stage could teach a model to form reasoning strategies. And at the inference stage — that is, the stage when the model is being used — more test-time compute could allow the model to generate longer chains of thought when answering complex questions, and thus continue improving performance.

Both changes amplify the effect of distillation. Large-scale distillation is a method mainly used in the post-training stage; as the importance of post-training rises, the return on investment from distillation also rises. At the same time, test-time compute allows the model, in addition to producing a final answer, to also produce long reasoning processes — chains of thought, tool calls, search processes, correction processes. These outputs can be used as raw material for distillation data.

Four months later, the distillation process disclosed by DeepSeek in the R1 technical report showed in more detail the mechanism by which distillation makes models stronger. R1 had several specific practices and findings.

The distillation process mainly used “question–reasoning process–answer” data pairs generated by R1. This is currently the ideal state for distillation, and it works better than data pairs that contain only question and answer.

When releasing R1, the highly open DeepSeek directly displayed the full chain of thought, saying it “hoped to help the community distill better small models.”

By contrast, leading closed-source model companies, including OpenAI, Anthropic, and Google DeepMind, have always hidden full chains of thought and reasoning traces from users. That is why practitioners often say, when talking about distillation, that “so-and-so cracked so-and-so’s chain of thought.”

This Monday, August 10, researchers from the University of Tübingen in Germany and other institutions released a paper titled “Stealing Reasoning Traces from Proprietary LLM APIs,” showing some methods they had found for reconstructing reasoning traces. The research website is stolen-thoughts.com.

In fact, this has not been a secret for a long time. Even if closed-source model companies deliberately hide them, chains of thought and reasoning traces can still be reconstructed through technical means. The essential reason is that they are all part of model output — products of the model usage stage. As long as you use a model, chains of thought and reasoning processes are generated and leave traces. This still belongs to black-box distillation and hard distillation.

Another finding from R1 about distillation is that in the post-training stage, direct distillation brings greater improvement than having the model do reinforcement learning on its own.

At the time, DeepSeek ran a controlled experiment using Qwen2.5-32B. After more than 10,000 steps of large-scale reinforcement learning on Qwen-32B-Base, the resulting model scored 47.0% on AIME 2024, a benchmark that evaluates mathematical problem-solving ability. After supervised fine-tuning Qwen-32B-Base with 800,000 pieces of data generated by R1, the resulting model scored 72.6% on AIME 2024, more than 25 percentage points higher than the RL method.

“We demonstrate that the reasoning patterns of larger models can be distilled into smaller models, resulting in better performance compared to the reasoning patterns discovered through RL on small models.”
— R1 technical report

Although this was the result of a specific experiment and may not generalize to all situations, it is still an attractive finding. Long-step reinforcement learning is inherently more difficult than supervised fine-tuning. It places more demands on infrastructure, is often slower, and has higher compute costs.

DeepSeek made public a relatively economical, efficient, and deterministic way to improve the reasoning ability of smaller or weaker models.

Several practitioners said that some recent practices mainly use supervised fine-tuning in the post-training stage, with little or almost no reinforcement learning, and can still achieve very good results.

It was also after DeepSeek-R1, throughout 2025 and up to now, that Anthropic, OpenAI, Google, and other companies said the “distillation attacks” they had detected were increasing.

During the same period, more exploration of post-training also promoted another form of distillation that is not controversial: on-policy distillation, currently mainly used for merging post-training capabilities.

The difference between on-policy and off-policy distillation lies in who generates the data. The distillation mentioned earlier, in which a model learns from another closed-source model, is mostly off-policy distillation: the data is generated by the teacher model. On-policy distillation, by contrast, has the student model generate reasoning and answers, while the teacher model provides feedback. The feedback can be the distribution probabilities generated token by token (white-box), or it can be a judgment of the reasoning trace and answer (black-box).

Since the second half of 2025, Alibaba Qwen, Thinking Machines Lab, and Xiaomi MiMo have all contributed practices and improvements in on-policy distillation.

In the technical report for MiMo V2-Flash early this year, Xiaomi introduced MOPD, or multi-teacher on-policy distillation. In June, it published a separate paper titled “Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training.” The method first trains teacher models in different directions — math, coding, tool use — and then has the student model generate its own trajectories and receive feedback from different teachers according to the task. It aims to solve a new post-training problem: if capabilities in multiple directions are directly mixed together for reinforcement learning, they often interfere with each other, with gains in one area coming at the expense of another.

The technical reports for DeepSeek-V4 and Kimi K3 both said that in the post-training stage, they used MOPD-like ideas to merge multiple expert models.

Using AI to accelerate AI

The third line of development in the scaling of distillation is that distillation itself is becoming more automated as AI capabilities grow stronger.

Return to the key of distillation: “question–reasoning process–answer” data pairs. The question-asking step can shift from humans to AI. Since 2022, studies such as Self-Instruct have tried to solve the problem that high-quality questions are scarce and expensive.

In practice, companies can first screen high-quality real questions from authorized user behavior, then expand from these real questions to generate more AI-created questions, which can be used to ask the teacher model more frequently and obtain more answers. It is like starting with some yeast — real data — and using it to ferment a larger dough.

The complete “question–reasoning process–answer” data pairs can also be rewritten and expanded in this way. This is essentially a basic way of thinking in today’s AI field: using AI and models to automate and accelerate AI itself.

Many specific steps in the distillation process can also use AI. These include selecting high-quality questions from massive amounts of real questions, evaluating what counts as high-quality data, and improving the diversity of synthetic data. Some steps can be handled by models. Others can be accelerated and optimized by increasingly powerful AI programming capabilities, which make it faster to build and improve various systems.

Since 2025, the various methods and practices of large-scale distillation have gradually matured. Its effects and necessity have also become more apparent as the importance of post-training has increased. Discussion around “distillation” has also gradually moved beyond the AI technical community. In the process of communication, it has been simplified, misunderstood, and even distorted. Distillation is no longer a purely technical issue. It has moved to the center of the storm.

Misunderstandings about distillation: it is neither a silver bullet nor a secret

As mentioned repeatedly above, distillation that makes large language models stronger mainly happens in the post-training stage, and also includes the mid-training stage. But a model’s overall effectiveness comes from the complete training process, from pre-training to post-training. Pre-training is generally believed to be more important. Distillation is not the most important factor determining a model’s performance.

In the DeepSeek-R1 technical report, the same 800,000 pieces of R1 data were used for distillation. After distillation, the Qwen2.5-32B base model scored 72.6% on AIME 2024, higher than the 70.0% scored by the distilled Llama-3.3-70B-Instruct, even though the latter had more than twice as many parameters.

After K3 was released, Ai2 researcher Nathan Lambert replied to a tweet about K3 topping the Frontend Code Arena ranking, which evaluates front-end coding ability. He said that by this point, the whole “distillation” narrative should stop; people should recognize that China is also very good at building models.

A certain level of pre-training is the foundation that allows Chinese open-source models such as K3, GLM-5.2, and DeepSeek-V4 to achieve their current performance.

Distillation is often compared to a shortcut. A shortcut usually implies ease and less effort. But in reality, carrying out distillation at large scale today is a relatively complex systems-engineering project.

Based on descriptions from several practitioners, large-scale distillation has several difficult parts.

First, a team must be able to call leading models stably, frequently, and at large volume, while also doing user operations. One practice mentioned is to build many relay sites and attract a group of specific users with real usage behavior through discounts or other methods. These users might be advanced programmers, or science and engineering students and researchers who need to handle a large number of research questions. In daily use, they naturally generate high-quality multi-turn questions from real scenarios and real tasks, and then receive model answers. After being screened and processed in certain ways, the data from these questions and answers can serve as the source for fermenting and expanding more data.

This tests a team’s operations capabilities, including whether it knows where high-quality users are and how to reach them. It also tests the engineering capability to build such a system. It also requires some ecosystem capabilities, because the whole process may require cooperation with third-party companies or institutions.

Second is the team’s own ability to construct high-quality questions and tasks. This requires deep understanding of the tasks, the data, and the performance boundaries of current leading models. It overlaps with some of the capabilities needed for large-model training itself.

Third is how to use the data well. This requires building a data pipeline: whether the task distribution is reasonable; how to sample, screen, filter, deduplicate, expand, and correct; and how to determine formats and ratios. This pipeline has some measurable optimization indicators, such as what proportion of the raw data obtained can ultimately be used in post-training, and how efficient and high-quality the expansion is. The quality of the data pipeline affects effectiveness, efficiency, and cost.

Continuous distillation is also very expensive. There are some circulating claims about what companies in the industry are spending and budgeting for distillation this year, ranging from more than $100 million to $1 billion.

One AI investor said: distillation is not a simple button. It is not that you click once and model performance suddenly leaps forward. There are many implementation problems involved. Distillation also requires calculating return on investment.

Is distillation a moat for a model team?

Most of the practitioners we spoke with, whether they came from companies rumored to be doing distillation or from companies that do not distill, held relatively consistent views: for first-tier companies, distillation does not constitute a long-term moat.

Like many technologies in large models, the ideas and practices of distillation will gradually spread. Personnel movement, open-source sharing, conference exchanges, and third-party service providers looking for more customers all contribute to this. People and information in the AI circle are constantly moving. Researchers repeatedly expressed a similar sentiment: there are no real secrets in the large-model field.

The competitive advantage brought by a technical method itself is mostly first-mover advantage. Those who do it first will have more experience, but it is not an insurmountable, winner-takes-all moat like a network effect.

The phenomenon truly seen as having a stronger competitive moat is the “data flywheel”: if one company’s model is strong enough and can reach large numbers of users who use it for difficult tasks, it will receive more high-quality data flowing back. These data are unique, non-public, and unavailable to others. After certain processing, these data can again be used to help train stronger models, which then attract more users to handle harder tasks.

In this flywheel logic, applications that directly contact users are highly valuable. For example, depending on user agreements and permission settings, applications such as Cursor, Devin, and Manus may be able to obtain more complete data and user behavior than the models they call.

Yesterday, August 12, after Grok 4.6 was released, Elon Musk (马斯克) replied to a tweet saying Devin had integrated Grok 4.6. He said: “Grok 4.7 will be better than all current models.” “The training corpus of SpaceX is so awesome and unique.” A $60 billion acquisition of Cursor seems to have been worth it.

The data flywheel also has its own controversies. Model and application companies can obtain data, but can they use those data for training? The more difficult the task and the higher-value the scenario, the more likely customers and users may be to prefer controlling those data themselves. At the same time, as the number of model users continues to expand, user types and scenarios become more diverse. Is it still worth searching for gold in the sand?

One practitioner believes that in some AI applications that lean toward life assistants, entertainment, and leisure, the vast majority of user-generated data are garbage for training stronger models.

The internal line of judgment: black box or white box

The most interesting phenomenon about distillation is this: no one wants to talk about it publicly, but most of the practitioners we spoke with do not believe deep down that it is a shameful practice or one that violates their technical convictions.

The dividing line in this internal judgment lies between black box and white box.

The distillation currently being done on closed-source models is all black-box distillation. The data it obtains are data generated during use after these models are released. They are products of the model as a product. Then why can’t other companies ask this model questions and obtain answers as users? Besides, everyone has paid real money for these questions and answers. Of course, in practice, various methods are used to “take advantage” and lower costs.

The more controversial part here is reasoning traces. Most models hide the full reasoning traces, and the distilling party needs to use some technical means to infer and reconstruct them. But reasoning traces are still products of the model usage stage.

Anthropic, Google, and OpenAI would say: my user agreement clearly states that other competitors cannot use my model to train and improve their own models.

But who was sued by The New York Times, accused of copying and using without permission the newspaper’s archive of journalism accumulated over more than 170 years by generations of reporters, commentators, and authors? OpenAI. Who downloaded huge numbers of books from pirate platforms, was unwilling to pay for any of them, was sued in a class action by several American authors, and just reached a $1.5 billion settlement? Anthropic.

Even some American AI practitioners believe companies such as Anthropic are being highly hypocritical. It is a bit like the situation in Christopher Nolan’s new film The Odyssey: you sent the Trojan Horse into Troy, and now your own homeland is being invaded by those who come from the sea.

Large-scale distillation is a new issue that has emerged after the development of new technology. It is hard to make most people sincerely accept that this is undesirable, unethical, or even shameful and evil just because several companies say it violates their user agreements. Moreover, violating a user agreement does not necessarily constitute legal infringement. This also involves other legal rules, jurisdictional questions, and more.

In the broadest sense, distillation — that is, using data to improve a model — has become ubiquitous.

A widely circulated way to verify distillation is actually invalid: when you ask a model “Who are you?” and Model A says it is Model B, this is not ironclad proof that A distilled B. In the pre-training stage, everyone uses large amounts of public internet data, and much of that data already includes content generated by various models themselves.

In July this year, Nvidia founder Jensen Huang (黄仁勋) was asked about distillation in an interview with Axios. He said: “Distillation, learning from AI and learning from other sources of knowledge, is the fundamental principle of intelligence.”

For most model companies, including some American model companies, the accelerator on distillation has already been pressed. Few will voluntarily give up distillation in the short term.

Is it really impossible for distillation to surpass the teacher model?

We also asked several practitioners this question. Their answers were similar: technically it is not impossible, but there are potential organizational risks.

Distillation has some inherent technical problems. For example, it may cause the student model to learn some of the teacher model’s errors, biases, refusal habits, and expression patterns.

But distillation is only one part of model training. There are many other parts of the complete model training process that can be improved: pre-training data, architecture, algorithms, infrastructure. If multiple optimizations are stacked together, is it possible for a student model to become better than a given teacher model?

Some studies have shown that on certain specific tasks, student models can surpass teacher models. In December 2024, for example, Microsoft released Phi-4, a 14-billion-parameter model. A large amount of its training data was synthesized by teacher models such as GPT-4o. Phi-4 surpassed GPT-4o on two benchmarks:

• GPQA, which evaluates graduate- and PhD-level scientific knowledge and reasoning ability: Phi-4 scored 56.1%, while GPT-4o scored 50.6%.

• MATH, which evaluates mathematical problem-solving and reasoning ability: Phi-4 scored 80.4%, while GPT-4o scored 74.6%.

What is special is that as a small-parameter model, Phi-4 also directly used data generated by teacher models in pre-training. But when training truly large-scale models with trillions of parameters, the pre-training stage requires enormous amounts of data, and it is difficult to use data obtained through distillation. Compared with directly processing web pages, code, and books, calling teacher models one by one to generate data is slow and expensive.

But this also raises a possibility: when model inference speed increases greatly and prices fall sharply, can distillation — or data constructed with the help of stronger models — enter pre-training more often?

At the same time, can ideas such as multi-teacher distillation also be used to make models stronger? In theory, a student model can learn from different strongest models such as Claude and GPT at the same time. This may bring new technical problems, such as training instability and interference between different capabilities when distilling from different base models.

More aggressive possibilities include whether the most leading companies can distill their own models to achieve a kind of “lifting themselves by stepping on their own foot.”

From a pure research perspective, “whether a model using distillation can surpass the teacher model” is at least a question that remains to be tested and explored.

Distillation is a relatively economical method that produces results quickly. An athlete can certainly both take stimulants and train diligently. But in reality, the two are often hard to balance, because luck and inertia set in.

When a team, during a certain stage, puts a relatively large share of its attention and resources into distillation, projects and individuals exploring more uncertain and longer-term directions may not receive enough resources or recognition.

When someone accuses you of doing something wrong, responding with “didn’t you do the same?” does not resolve the conflict.

At this moment, open-source models are increasingly approaching the leading closed-source models in performance. It is also an objective fact that the strongest open-source models come from China, while the strongest closed-source models come from the United States.

Earlier, Zhipu had already been placed on the U.S. Commerce Department’s Entity List in January 2025. The Intelligence Authorization Act for Fiscal Year 2026, which took effect at the end of the same year, requires DeepSeek to be removed from U.S. intelligence systems, national security systems, and related suppliers.

By April this year, the U.S. House Committee on Homeland Security and the Select Committee on the Strategic Competition Between the United States and the Chinese Communist Party had begun investigating American companies’ use of Chinese models. They asked Cursor why it used Kimi K2.5 as the base for Composer, and asked Airbnb why it used Alibaba Qwen in its customer service business.

After Kimi K3 was released on July 16, the U.S. government was reportedly considering restricting or even banning some Chinese open-source models. On July 24, 77 companies and organizations, including Microsoft, Nvidia, Meta, and Fireworks AI, successively signed the open letter “Open Weights and American AI Leadership,” opposing rushed restrictions on open-source models.

Jensen Huang’s first post on Twitter was sharing this open letter.

New restrictive measures in recent months are still under discussion and have not yet been implemented.

Companies including Anthropic, OpenAI, and Google are also taking stricter technical measures to identify and ban accounts suspected of being used for distillation. Anthropic, for example, said it had built classifiers and behavioral fingerprinting systems to identify distillation traffic, detecting cross-account coordination, repeated questioning, and attempts to extract chains of thought. It will also strengthen identity verification for education, research, and startup accounts.

In the University of Tübingen “stealing reasoning traces” study mentioned earlier, the methods used to reconstruct reasoning traces had already been reported by the research team to the relevant closed-source model companies. By the time the researchers uploaded the paper, some of those methods had already stopped working.

All parties involved are making their own choices and preparations. The changes that follow will affect the entire industrial chain, from compute, cloud services, and infrastructure to models, applications, and customer deployment.

The storm is still continuing. The next eye of the storm may not necessarily be distillation. Distillation itself is only one method for optimizing models.

So many companies are investing so many resources and rushing into this intelligence race. What are they doing it for? In the first half of this year, explosive growth in coding and agents reversed market expectations. Further ahead, if the scale and speed of large-model application adoption cannot continue, how will the future of the model R&D race fluctuate? That is a question some others are already watching.