2018
Alec Radford
Generative Pre-Training
With colleagues, showed how generative language pre-training could support downstream tasks, establishing the original GPT approach.
The 2018 report Improving Language Understanding by Generative Pre-Training credited Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever.
Radford later co-led work including CLIP, connecting text and images through contrastive training. GPT's early research should be distinguished from the later public launch of ChatGPT.
Dated contributions
2018
Generative Pre-Training
Alec Radford
With colleagues, showed how generative language pre-training could support downstream tasks, establishing the original GPT approach.
Report · 2018 — Improving Language Understanding by Generative Pre-Training ↗2021
Learning from Images and Text
Alec Radford, Jong Wook Kim & collaborators
CLIP connects natural-language descriptions with visual representations through contrastive learning.
Paper · 2021 — Learning Transferable Visual Models From Natural Language Supervision ↗
Reference
Original work & historical records
- 2018 · Report — Improving Language Understanding by Generative Pre-Training ↗
- 2021 · Paper — Learning Transferable Visual Models From Natural Language Supervision ↗
