Paper Reading List¶

Step1: NMT, Attention and Transformer¶

TODO: 2014-Google-Sequence to Sequence¶

Sequence to Sequence Learning with Neural Networks

TODO: 2014-Bengio-Attention¶

Neural Machine Translation by Jointly Learning to Align and Translate. 首次提出了Attention的概念

TODO: 2017-Transformers¶

Attention Is All You Need

Step2: GPT Series¶

2018-OpenAI-GPT1.0¶

Improving Language Understanding by Generative Pre-Training

2018-Google-BERT¶

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

2019-OpenAI-GPT2.0¶

Language Models are Unsupervised Multitask Learners

2020-OpenAI-GPT3.0¶

Language models are few-shot learners

Step3: Engineering in LLM¶

TBD

Reference¶

• Understanding Large Language Models -- A Transformative Reading List
• Awesome-LLM/Milestone Papers