Evolution of Model Architectures A practical timeline of neural-network architecture, from the perceptron and …
Long Short-Term Memory (LSTM): Hochreiter and Schmidhuber (1997) How LSTM cells use gated state to learn long-range sequence dependencies, and …
Backpropagation: Rumelhart, Hinton, and Williams (1986) How backpropagation uses the chain rule to train multilayer neural networks, and …
The Perceptron: Rosenblatt (1958) How Rosenblatt's perceptron learns a linear decision boundary, why convergence …
Encoder-Decoder Models and Seq2Seq Learning How encoder-decoder models map one variable-length sequence to another, from the …
Transformers: Attention Is All You Need (2017) How the 2017 Transformer replaces recurrence with self-attention, including its …
Attention Mechanisms: Bahdanau, Cho, and Bengio (2014) How neural attention scores, weights, and combines input representations, from …
GPT Models: Architecture and Training How GPT combines causal Transformers, next-token prediction, scale, prompting, …
Convolutional Neural Networks: LeNet-5 and LeCun et al. (1998) How convolution, shared weights, and pooling made LeNet-5 practical for image …
The Turing Test: Computing Machinery and Intelligence (1950) What Turing's 1950 imitation game actually tests, the objections the paper …
