Deep dives and research notes on deep learning, probability, and the mathematics underneath.https://ritwiksinghai.com/en-ushttps://ritwiksinghai.com/articles/what-attention-computes/https://ritwiksinghai.com/articles/what-attention-computes/Attention is a soft, differentiable dictionary lookup—dot-product scores, a softmax, and a weighted sum of values—that gives every position a global receptive field in a single layer.Wed, 20 May 2026 00:00:00 GMTAttentionTransformersDeep LearningLinear AlgebraSequence Modelshttps://ritwiksinghai.com/articles/gradient-descent/https://ritwiksinghai.com/articles/gradient-descent/Gradient descent trains almost every neural network by repeatedly taking a small step opposite the direction of steepest ascent — a procedure whose successes and pathologies both follow from that single geometric idea.Wed, 22 Apr 2026 00:00:00 GMToptimizationgradient descentdeep learningcalculusmachine learninghttps://ritwiksinghai.com/articles/central-limit-theorem/https://ritwiksinghai.com/articles/central-limit-theorem/The sample mean of many independent finite-variance draws becomes Gaussian, and its spread shrinks like σ/√n — a rate that follows from how variance adds, and that breaks precisely when its hypotheses do.Sat, 28 Mar 2026 00:00:00 GMTprobabilitycentral-limit-theoremstatisticsasymptoticsgaussianhttps://ritwiksinghai.com/articles/sensitive-dependence/https://ritwiksinghai.com/articles/sensitive-dependence/Why fully deterministic systems—weather, pendulums, a tumbling die—can still defy long-range prediction, and how a positive Lyapunov exponent turns finite-precision measurement into vanishing knowledge of the future.Tue, 03 Mar 2026 00:00:00 GMTchaosdynamical systemsLorenzpredictionnonlinear dynamicshttps://ritwiksinghai.com/articles/sum-of-circles/https://ritwiksinghai.com/articles/sum-of-circles/Any reasonable periodic function is a sum of rotating vectors, and the recipe for the radii is a sequence of inner products.Mon, 09 Feb 2026 00:00:00 GMTFourierSignal ProcessingMathematicsHarmonic Analysishttps://ritwiksinghai.com/articles/the-golden-angle/https://ritwiksinghai.com/articles/the-golden-angle/A sunflower head packs its seeds by placing each one a fixed angle of about 137.5° from the last, and that exact angle falls out of the golden ratio being the hardest number to approximate by fractions.Sun, 18 Jan 2026 00:00:00 GMTphyllotaxisgolden-ratiocontinued-fractionsfibonaccigeometryhttps://ritwiksinghai.com/logs/attention-paper/https://ritwiksinghai.com/logs/attention-paper/Research log — The Evolution of AttentionMon, 15 Dec 2025 00:00:00 GMTNLPFoundationsDeep Learninghttps://ritwiksinghai.com/articles/rag-failure/https://ritwiksinghai.com/articles/rag-failure/The final abstraction layer of any intelligent system is limited to the language English. Why???Mon, 20 Oct 2025 00:00:00 GMTLLMLanguageMathematicsPhilosophy