Evolution of Model ArchitecturesA practical timeline of neural-network architecture, from the perceptron and …
Long Short-Term Memory (LSTM): Hochreiter and Schmidhuber (1997)How LSTM cells use gated state to learn long-range sequence dependencies, and …
Backpropagation: Rumelhart, Hinton, and Williams (1986)How backpropagation uses the chain rule to train multilayer neural networks, and …
The Perceptron: Rosenblatt (1958)How Rosenblatt's perceptron learns a linear decision boundary, why convergence …
Encoder-Decoder Models and Seq2Seq LearningHow encoder-decoder models map one variable-length sequence to another, from the …
Transformers: Attention Is All You Need (2017)How the 2017 Transformer replaces recurrence with self-attention, including its …
Attention Mechanisms: Bahdanau, Cho, and Bengio (2014)How neural attention scores, weights, and combines input representations, from …
GPT Models: Architecture and TrainingHow GPT combines causal Transformers, next-token prediction, scale, prompting, …
Convolutional Neural Networks: LeNet-5 and LeCun et al. (1998)How convolution, shared weights, and pooling made LeNet-5 practical for image …
The Turing Test: Computing Machinery and Intelligence (1950)What Turing's 1950 imitation game actually tests, the objections the paper …