Definition
An extension of the AdaGrad optimization algorithm that dynamically adapts learning rates based on a moving window of past gradient updates to prevent excessive decay.
In short: An extension of the AdaGrad optimization algorithm that dynamically adapts learning rates based on a moving window of past gradient updates to prevent excessive decay.