Timeline [16]
From the rule for updating belief to the architecture behind modern language models — the through-line the atlas follows.
Thomas Bayes (posthumously, communicated by Richard Price); generalized by Pierre-Simon Laplace
Bayes' essay, published in 1763 after his death, gave a rule for updating the probability of a hypothesis given evidence; Laplace independently restated and developed it into the general form in the following decades.
Joseph Fourier
Fourier claimed (in an 1807 memoir, expanded in his 1822 treatise on heat) that any periodic function can be represented as a sum of sines and cosines, founding harmonic analysis and the spectral methods central to signal processing.
Carl Friedrich Gauss and Adrien-Marie Legendre; Laplace
Legendre published least squares (1805) and Gauss tied it to the normal error distribution and gave a probabilistic justification (1809); Laplace's 1812 treatise unified these with the central limit theorem, establishing the statistical foundations of regression.
Andrey Markov
Markov introduced chains of dependent random variables in which the next state depends only on the current state, providing the formalism later used in HMMs, MCMC sampling, and reinforcement learning.
Warren McCulloch and Walter Pitts
They proposed a mathematical model of a neuron as a threshold logic unit, showing networks of such units can compute logical functions and laying the conceptual basis for neural networks.
Frank Rosenblatt
Rosenblatt introduced the perceptron, a trainable linear classifier with a learning rule that adjusts weights from labeled examples, the first practical learning algorithm for a neural model; its linear-separability limits were later analyzed by Minsky and Papert (1969).
Edward Lorenz
While studying a simplified model of atmospheric convection, Lorenz found that deterministic nonlinear systems can exhibit sensitive dependence on initial conditions, founding chaos theory and clarifying limits of long-term prediction.
Seppo Linnainmaa
Linnainmaa described the efficient reverse accumulation of derivatives through a computation graph, the algorithm that underlies backpropagation and all modern gradient-based deep learning frameworks.
David Rumelhart, Geoffrey Hinton, and Ronald Williams
Their Nature paper demonstrated that backpropagation could train multilayer networks to learn useful internal representations, reviving neural network research after the perceptron's limitations.
Yann LeCun and collaborators
LeCun applied backpropagation to convolutional networks for handwritten digit recognition (1989), culminating in LeNet-5 (1998), which used weight sharing and local receptive fields to exploit image structure.
Sepp Hochreiter and Jurgen Schmidhuber
LSTM introduced a gated memory cell that mitigates the vanishing-gradient problem, enabling recurrent networks to learn long-range dependencies in sequences.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton
A deep convolutional network trained on GPUs won the ImageNet ILSVRC-2012 competition by a large margin, igniting the modern deep learning era in computer vision.
Tomas Mikolov and colleagues at Google
word2vec learned dense vector embeddings of words via the skip-gram and CBOW objectives, capturing semantic relationships as linear structure (e.g. king - man + woman ~ queen) and popularizing distributed word representations.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio
They added a soft attention mechanism to encoder-decoder RNNs, letting the decoder dynamically weight source positions and removing the fixed-length bottleneck, the conceptual precursor to the Transformer.
Ian Goodfellow and colleagues
GANs frame generative modeling as a minimax game between a generator and a discriminator, producing high-fidelity samples without an explicit likelihood.
Ashish Vaswani and colleagues at Google
The paper 'Attention Is All You Need' replaced recurrence and convolution with pure self-attention, enabling massive parallel training and becoming the backbone of subsequent large language models such as BERT (2018) and the GPT series (2018-).