DeepSeek is under fire after its AI chatbot R1 shook up the market earlier this year. The small Chinese firm claimed R1 matches the giants’ AI performance while using far less computing power and cash. That sent Western tech stocks into a freefall. Nvidia lost more market value in a single day than any company ever.
The heat came when sources alleged DeepSeek ripped off OpenAI’s secret o1 model through a method called distillation. The press framed this as a potential game-changer, suggesting DeepSeek had found a new way to build AI.
But distillation is actually an established AI practice. Enric Boix-Adsera, a researcher at Wharton, said: 
Distillation is one of the most important tools that companies have today to make models more efficient.
Distillation dates back to a 2015 Google paper by Geoffrey Hinton and others. They found big model ensembles could be compressed into smaller “student” models using “soft targets” from large “teacher” models. This “dark knowledge” speeds up learning and slashes costs.
“Wrong answers were all considered equally bad,” said Oriol Vinyals, co-author at Google.
“The teacher model revealed that some mistakes are closer than others.”
Early on, distillation was ignored. But as models exploded in size, it gained traction. Google’s 2018 BERT was cut down to DistilBERT in 2019, making it cheaper and faster — a practice now common at Google, OpenAI, and Amazon.
DeepSeek’s alleged copy of OpenAI’s o1 model via distillation is unlikely. Distillation needs inside access to the teacher model, which isn’t possible with closed-source systems. Instead, models can learn indirectly by prompting answers — a “Socratic” style of distillation.
Berkeley’s NovaSky lab recently showed distillation could train complex chain-of-thought AI models for under $450, matching much larger models. Co-lead Dacheng Li said:
“We were genuinely surprised by how well distillation worked in this setting.”
“Distillation is a fundamental technique in AI.”
DeepSeek sparked the noise, but distillation is an old, proven tool powering many state-of-the-art AI systems today.
