Attention Mechanisms: Bahdanau, Cho, and Bengio (2014)
How neural attention scores, weights, and combines input representations, from …
Architectures
Backpropagation: Rumelhart, Hinton, and Williams (1986)
How backpropagation uses the chain rule to train multilayer neural networks, and …
Convolutional Neural Networks: LeNet-5 and LeCun et al. (1998)
How convolution, shared weights, and pooling made LeNet-5 practical for image …
Encoder-Decoder Models and Seq2Seq Learning
How encoder-decoder models map one variable-length sequence to another, from the …
Evolution of Model Architectures
A practical timeline of neural-network architecture, from the perceptron and …
GPT Models: Architecture and Training
How GPT combines causal Transformers, next-token prediction, scale, prompting, …
Long Short-Term Memory (LSTM): Hochreiter and Schmidhuber (1997)
How LSTM cells use gated state to learn long-range sequence dependencies, and …
The Perceptron: Rosenblatt (1958)
How Rosenblatt's perceptron learns a linear decision boundary, why convergence …
The Turing Test: Computing Machinery and Intelligence (1950)
What Turing's 1950 imitation game actually tests, the objections the paper …
Transformers: Attention Is All You Need (2017)
How the 2017 Transformer replaces recurrence with self-attention, including its …