Rodríguez Abella, Álvaro Silvestre, Joao Pedro Tabuada, Paulo 2026-05-18T13:34:55Z 2026-05-18T13:34:55Z 2025-05-01 2640-3498 http://hdl.handle.net/11531/110122 Artículos en revistas . A key component of transformers is the attention mechanism orchestrating how each token influences the propagation of every other token along the layers of a transformer. In this paper we provide a rigorous, mathematical analysis of the asymptotic properties of attention in transformers. Although we present several results based on different assumptions, all of them point to the same conclusion, all tokens asymptotically converge to each other, a phenomenon that has been empirically reported in the literature. Our findings are carefully compared with existing theoretical results and illustrated by simulations and experimental studies using the GPT-2 model. application/pdf en-GB Creative Commons Reconocimiento-NoComercial-SinObraDerivada España http://creativecommons.org/licenses/by-nc-nd/3.0/es/ Revista: Proceedings of Machine Learning Research, Periodo: 1, Volumen: , Número: 267, Página inicial: 174, Página final: 184 Consensus is all you get: the role of attention in transformers info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/openAccess . transformers; attention mechanism; token convergence; asymptotic analysis.