2018

Alec Radford

Generative Pre-Training

With colleagues, showed how generative language pre-training could support downstream tasks, establishing the original GPT approach.

The 2018 report Improving Language Understanding by Generative Pre-Training credited Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever.

Radford later co-led work including CLIP, connecting text and images through contrastive training. GPT's early research should be distinguished from the later public launch of ChatGPT.

Dated contributions

  1. 2018

    Generative Pre-Training

    Alec Radford

    With colleagues, showed how generative language pre-training could support downstream tasks, establishing the original GPT approach.

    Report · 2018 — Improving Language Understanding by Generative Pre-Training ↗
  2. 2021

    Learning from Images and Text

    Alec Radford, Jong Wook Kim & collaborators

    CLIP connects natural-language descriptions with visual representations through contrastive learning.

    Paper · 2021 — Learning Transferable Visual Models From Natural Language Supervision ↗

Reference

Original work & historical records

Wikipedia