Transformer models: an introduction and catalog

02/12/2023

∙

by   Xavier Amatriain, et al.

∙

0

∙

share

In the past few years we have seen the meteoric appearance of dozens of models of the Transformer family, all of which have funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovation in Transformer models.