Definition
An initial phase of training where the learning rate is gradually increased from a very small value to a target maximum, which helps stabilize the early training of deep neural networks.
In short: An initial phase of training where the learning rate is gradually increased from a very small value to a target maximum, which helps stabilize the early training of deep neural networks.