Gradient descent on a quadratic you can solve exactly
Solve gradient descent on a scalar quadratic exactly, identify the stable learning-rate interval, and distinguish convergence from oscillation.
Foundation
The journal / Ideas in progress
Engineering & everyday learningNotes from
the work.
Software, systems, and the things I’m learning along the way. From first principles to AI in production.
Find something to readAll notes 6
A little theory. A lot of practice.
Split by the person when the person is the prediction boundary
Compare lookup errors under recording and speaker splits, then construct connected blocks that keep both speakers and devices separate.
Foundation
Check a gradient with finite differences before trusting training
Derive central-difference error, expose floating-point cancellation and a hidden gradient bug, and test the limits of directional checks.
Foundation
Likelihood is a function of the parameter
Derive a Bernoulli likelihood, compare sample-size effects and serial dependence, and reproduce why equal success counts need not mean equal sequence probabilities.
Foundation
Expected loss and variance from a tiny discrete distribution
Calculate expected loss, variance and the distribution of a two-case average, then test how copied observations and rare large failures change the evaluation.
Foundation
Read residuals before adding another model
Fit a tiny linear model, recognize curvature in its residuals, and separate misspecified mean structure from changes in error spread.
Foundation