Definition
A model compression technique where a smaller, more efficient neural network (the "student") is trained to mimic the complex, learned output probabilities of a larger, high-performing model (the "teacher").
In short: A model compression technique where a smaller, more efficient neural network (the "student") is trained to mimic the complex, learned output probabilities of a larger, high-performing model (the "t...