2017

Vaswani et al.

2017

Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser and Illia Polosukhin introduced the Transformer.

Their 2017 paper Attention Is All You Need credited all eight authors. The architecture used attention rather than recurrence to connect sequence elements, enabling parallel training.

Transformers became the basis of GPT and many other language and multimodal models. This was a collaborative research achievement, not a sole invention by the pictured lead author.

Dated contributions

  1. 2017

    Attention Is All You Need

    Vaswani et al.

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser and Illia Polosukhin introduced the Transformer.

    Paper · 2017 — Attention Is All You Need ↗

Reference

Original work & historical records

Wikipedia