Definition
A mechanism within neural networks, especially Transformers, that allows the model to dynamically weigh and aggregate information from different positions in an input sequence.
In short: A mechanism within neural networks, especially Transformers, that allows the model to dynamically weigh and aggregate information from different positions in an input sequence.