Definition
An optimization algorithm that adapts the learning rate for each parameter, performing smaller updates for frequently occurring features and larger updates for infrequent ones.
In short: An optimization algorithm that adapts the learning rate for each parameter, performing smaller updates for frequently occurring features and larger updates for infrequent ones.