Distillation is a technique where an older “teacher” AI model is used to train a newer, “student,” model that replicates the capabilities of the earlier system — often at a much lower cost than producing an original model from scratch. Some forms of distillation are widely accepted and even encouraged by AI labs, such as when companies create smaller, more efficient versions of their own models, or allow outside developers to use distillation to build non-competitive technologies.
Basically one AI asks questions and records the answers to recreate training data without massive compute for training data.
https://www.latimes.com/business/story/2026-04-07/china-is-copying-u-s-ai-models-american-companies-say-it-is-costing-them-billions-of-dollars
Basically one AI asks questions and records the answers to recreate training data without massive compute for training data.
I see and agree with you then. I’m glad it gets us better cheap models but more traditional innovation would definitely be preferred.