Gradient descent on a quadratic you can solve exactly
Solve gradient descent on a scalar quadratic exactly, identify the stable learning-rate interval, and distinguish convergence from oscillation.
Foundation
The journal / Ideas in progress
Engineering & everyday learningNotes from
the work.
Software, systems, and the things I’m learning along the way. From first principles to AI in production.
Find something to readAll notes 11
A little theory. A lot of practice.
A heading outline that survives a visual redesign
Build a useful heading hierarchy for an article, keep typography in CSS, and verify the complete outline after layouts and components are composed.
Foundation
Split by the person when the person is the prediction boundary
Compare lookup errors under recording and speaker splits, then construct connected blocks that keep both speakers and devices separate.
Foundation
Check a gradient with finite differences before trusting training
Derive central-difference error, expose floating-point cancellation and a hidden gradient bug, and test the limits of directional checks.
Foundation
Language tags and direction belong in the document
Set language and direction accurately, isolate user-supplied text, and test mixed English, Vietnamese and Arabic content without reversing stored strings.
Foundation
Likelihood is a function of the parameter
Derive a Bernoulli likelihood, compare sample-size effects and serial dependence, and reproduce why equal success counts need not mean equal sequence probabilities.
Foundation
Expected loss and variance from a tiny discrete distribution
Calculate expected loss, variance and the distribution of a two-case average, then test how copied observations and rare large failures change the evaluation.
Foundation
Progressive enhancement for a search form
Build a shareable GET search before adding asynchronous results, with consistent server filtering, history behavior and keyboard feedback.
Foundation
Read residuals before adding another model
Fit a tiny linear model, recognize curvature in its residuals, and separate misspecified mean structure from changes in error spread.
Foundation
Backpropagation adds every path through a computation
Trace a shared scalar computation, reproduce its gradients, expose a detached branch and distinguish a ReLU kink from a gradient bug.
Foundation
Use a table when readers compare values
Mark row and column headers in a plan comparison, preserve units and accessible names, and keep wide tables usable on narrow screens.
Foundation