Transformer

The transformer is the neural network architecture introduced in the 2017 paper Attention Is All You Need by researchers at Google. The architecture replaced earlier sequence modeling approaches with a mechanism called self-attention — allowing the model to weigh the relevance of every token in a sequence to every other token in parallel. The transformer architecture underlies essentially all modern large language models and is the technical foundation of the answer-engine era.

← Back to Glossary